Get up and running Zentral on Google Cloud Platform — Chapter 1
We start our tutorial series with a detailed “how to get up and running Zentral” guide. In four sequential blog posts we will run a Zentral server hosted at a public cloud provider, we edit Kibana and the Zentral base configuration, show how to inspect inner process (helps for debugging) and how Prometheus 2.0 is used in Zentral. We also enable 2FA login and wrap up the series with an example how to enable SAML login into Zentral (for the ones using Okta, OneLogin, et al.).
An brief overview of this tutorial series:
- Chapter 1 — GCP Project setup, zentral-all-in-one image, DNS
- Chapter 2— Start Zentral instance, setup Kibana 6.x index, 2FA login
- Chapter 3 — Process inspection, Prometheus 2.0 and base.yml conf
- Chapter 4 — Enable SAML login to Zentral
Zentral — a centralized server solution to gather, process, and monitor system events and link them to an inventory.
Zentral on Google Cloud Platform
For this tutorial series we run our setup on Google Cloud Platform (which we’ll abbreviate as “GCP”). The reasoning is quite simple — first GCP is easy to start with, pretty straightforward to use, and comes with great APIs and access control. But most importantly Google Cloud Platform provides an excellent written documentation to get started quickly and stay on track.
Of course there are alternative deployment methods available for Zentral (docker-compose, etc.). But for this tutorial series we focus primarily on a pre-build “Zentral-all-in-one” image. Everything we’ll discuss here based on GCP could be equally to apply to run Zentral on Amazon AWS cloud services. So in case you prefer to run on AWS, you need to adapt the initial steps for a similar “Zentral-all-in-one” setup. We have previously covered both platform providers in video tutorials, so start here to review the AWS specific setup steps when needed.
Tip: If you like follow our Chapters 1–4 in a ‘step by step’ approach by starting the free trial on Google Cloud Platform . Google gives you a nice ~$300 free-virtual-credit to get started in your journey using GCP. The free tier enables you to run a full blown Zentral-all-in-one server for a few weeks without paying a penny. So you can use this tutorial as great intro to start working with some GCP services.
Note: For GCP free trial you must provide credit card details. However your credit card will not be charged if you exhaust your $300 credit. You will be notified first.
Zentral-all-in-one
In case you’ll wonder what the “Zentral-all-in-one” thing is we’ve mention few times ? Well the answer is simple, we’ve just create a full stack VM to run Zentral which already contains all major components required.
A Zentral-all-in one install enables you to start evaluation quickly, become productive, and also have fun — just think about other dubbed “…in a box” projects here (i.e. Munki in a box ). With a running “Zentral-all-in-one” you can quickly start to run and control OSQuery, Google Santa. Just start to deploy both tools with your favourite fleet management tool to some test clients. We provide a auto-build process to create the TLS config and can (optionally) include latest OSQuery and Santa builds from GitHub baked into a single installer .pkg file. You can also explore additional integrations for Zentral, i.e. connect with Jamf Pro, Munki, SimpleMDM, Watchman, and do a lot more.
Some basic facts about the “Zentral-all-in-one” image:
We use HashiCorp Packer to create the image and upload to GCP and AWS. The VM build is based on generic Ubuntu 16.04.03 LTS. In there we bundle best breed of open source components along open source Zentral.
A short overview of these components used:
- Latest Zentral codebase from GitHub
- Django based Web-Server with Bootstrap based UI
- Nginx as reverse proxy
- ElasticSearch 6.1
- Kibana 6.1
- Prometheus 2.0
- Last but not least our special tooling in
/home/zentral/app/utilswhich help simplify setup from LetsEncrypt TLS configuration setup, create the initial (super)user, and run code updates.
Note: The contained open source components will progress independently of Zentral. Best consult the official documentation as your guide and learn about caveats involved for the specific tool in scope (i.e. migrate ElasticSearch from 5.x to 6.x requires some extra work up to re-indexing).
Updated documentation: https://zentral.readthedocs.io/en/latest/deployment/zaio-gcp/
2020 — see the updated tutorial: https://youtu.be/jsTMbfZQg8c
Stage 1 — Setup a GCP Project
If you decide to follow our GCP setup, starte here. In case you decide to run on AWS see our note above.
For Google Cloud Platform you need to adhere few base requirements:
- You need a Google account
- You need access to Google Cloud. If you don’t have just enroll into the free tier
- You need to have
gcloudCLI tool as part of Google Cloud SDK installed on your admin computer - You need to have the compute component API enabled in APIs & services section in GCP
- You should be able to create a DNS A record for a domain (you own)
Ok so once the base requirement is met, let’s start. As it’s good practice, we first create a dedicated project on GCP, we’ve here named our Project “Zentral-Demo”.
Next with the Google Cloud SDK installed we to run the gcloud CLI tool in Terminal — to get permission for interacting with the GCP API:
gcloud initOnce you’ve start the gcloud init command in Terminal, it asks to authenticate via a browser.
As you know from other Google service to log in, you may need to fill in your credentials or select the matching account profile.
Now we pick our “Zentral-Demo” project (by use of a numeric value), alternatively you could create a new Project via gcloud in Terminal:
Next is a critical step to follow — we need to import the “zentral-all-in-one” image from it’s public source-uri (this our Packer pre-build image). For that step just use the command below:
# new updated version (march 2019)
gcloud compute images create zentral-all-in-one \
--source-image zentral-all-in-one \
--source-image-project zentral-open-source
# deprecated version
gcloud compute images create zentral-all-in-one --source-uri gs://zentralopensource/zentral-all-in-one.tar.gz
You will see the import requires few minutes for the image copied over to your GCP project space:
Once that import finished, the “zentral-all-in-one” image is ready to create an instance.
Great so now you should be ready to set up your Zentral server. With having your GCP access prepared, our detailed steps above will take just approx. 10minutes, most of this time is to wait for the image copy over.
Attention : In case you happen to see an error like:
ERROR: (gcloud.compute.images.create) Could not fetch resource:
- Project 09991771727 is not found and cannot be used for API calls. If it is recently created, enable Compute Engine API by visiting…
You should follow instructions to enable API access for your user and project in GCP.
Stage 2 — Setup a GCP Compute instance
Select checkbox for the “zentral-all-in-one” image, then click on the “Create Instance” button.
We fill in a name for the VM, then select the zone (aka geographic location of the datacenter next to you) for the instance to run.
As we perspectively prepare for productive work we should change our Boot Disk size accordingly. The image default size is just 10GB, this is enough to boot from but of course not enough space to store much data in mid term. In the GCP web UI we extend Boot Disk size to a 250 GB standard persistent disk (optionally you can use SSD type with better IOPS performance). The key takeaway here…having more disc space is essential for ElasticSearch index data retention and may help with future maintenance tasks with Elastic Stack (i.e. re-index ElasticSearch indices).
Next thing we need to address is our instance firewall setting. We need to enable both HTTPS and HTTP. Reason to use both is we’re going to utilize the Let’s Encrypt service as our CA to acquire a free 90day TLS certificate, that can be renewed on ongoing basis (LE requires validation to TCP port 80).
Next once the VM instance has started we’ll see the (public) external IP here. It’s critical to copy the IP and create a public DNS A record. This is another mandatory step - with servers in general, if things go wrong…it’s always DNS.
Our favourite DNS provider is DNSimple . You need to find your equivalent service, probably in the DNS section from your domain provider of choice. Again what we do here is nothing exceptional special for Zentral. As with any other server/service you want create an DNS A record and set a FQDN to resolve to the external IP of your GCP instance.
Wrap up
Ok we’ve reached the end of this chapter. We have setup a project in GCP, we did setup our (mandatory) DNS A record, and we have prepared everything to startup Zentral next . As we we’re ready to move on see how startup the Zentral instance, setup Kibana index, and enable (optional) 2FA login in Chapter 2.
