My first Hackathon: bring Spinnaker to my company
I’ve joined my first Hackathon and worked on a project about using Spinnaker as CI/CD tool within company. The biggest challenge is to install Spinnaker on CentOS 7 with docker-compose.
Why Spinnaker?
- Spinnaker is dedicated to deploy services across multiple cloud providers and the integration with AWS, GCP, Azure is out of box.
- It’s focused on deploy stably, support full control of workflow, developers can customize the deployment flow to improve the quality of deployment, also it’s automatic.
- You will have a Web UI.
How to use Spinnaker?
Spinnaker can integrate with VCS like Github, Gitlab, Gitbucket, docker registry.
For example the workflow of deploy golang github project to Kubernetes cluster.
- A developer pushes code to Github repository, it triggers the Travis-CI that starts the test, build, package stages, a docker image is pushed to Docker repository finally.
- Spinnaker has a trigger on the Docker repository and starts to deploy that image to a Kubernetes cluster in some cloud provider.
- The deployment workflow includes Deploy to staging, Canary Deployment, Deploy to production.
Details of this hackathon
Our goal is to start a Spinnaker cluster and create a pipeline that can deploy the Git project to AWS and our own Kubernetes cluster.
There’re some contraints for this hackathon, I’m still new to the company so maybe it’s because of my limited information.
- I don’t have a Kuberntes cluster with API server’s access right, that means I can’t deploy Spinnaker cluster to Kubernetes.
- I don’t have a Debian/Ubuntu machine, that means I can’t start Spinnaker on a bare metal with Halyard.
- The Spinnaker need to be deployed within company’s infrastructure which is not any cloud provider, otherwise it can’t communicate with other infrastructure.
And what does Spinnaker need? It needs Halyard to configure the Spinnaker cluster, it will control where Spinnaker cluster is deployed, as we know Spinnaker itself is composed of microservices, hard to deploy on bare metal. Especially for its cloud provider settings, so Spinnaker relies on Halyard heavily, and every time the Spinnaker cluster update and upgrade can be managed safely.
Sadly, Halyard is only used for deploying Spinnaker itself to Kubernetes or some cloud providers and Debian/Ubuntu single machine. Then according to our constraints above, we lose every chance to use that tool. Maybe it’s time to give up :(
We didn’t give up, choose to install Spinnaker cluster the hardest way. Install Spinnaker cluster on Bare metal machine which is CentOS 7, for running the cluster easier, we use Docker to start each components.
My colleague found docker-compose.yml in Spinnaker, but the latest modification time was 4 years ago. After trying so many times we still couldn’t make it, I tried to give up actually, and also it was far away from quickstart and wasted so much time.
Actually I didn’t give up, I returned home and thought if the Spinnaker cluster can be deployed in Kubernetes, it can be deployed with Docker compose, since they didn’t have any difference.
So I chose the most stupid method, I created a Spinnaker cluster on GCP with far less time and rewrote the docker-compose.yml according to:
- Check each Kubernetes Pod’s yaml file to get the environment variables, image name, arguments…
- Login to the containers to fetch the mounted configuration files like clouddriver.yml, front50.yml, gate.yml… Yes, they’re generated by Halyard where the comment says
DONT'T EDIT THIS FILE, IT'S AUTOGENERATED BY....
After another X hours, without ignoring every details, I migrated all the Kubernetes configuration to docker-compose.yml, the second when I ran the docker-compose up -d and every microservice was labeled DONE and http://localhost:9000 worked.
Reflection of this hackathon
Yes, it’s not a happy ending for my first hackathon, our team didn’t reach our goal, there’s some experience for this time.
Gain
- The hardest way to know how the Spinnaker configuration works, how it deploys.
- I felt some joy when the installation was successful :)
- Some knowledge about the GCP (first time to use GCP)
Improvements next time
- Team member can be closer, whatever the seat we sit or the communication channel we talked, time is presious, we need check our progress frequently.
- Fail fast, although this time I worked out the installation, but it was not a great result, next time we should help the team member to tackle the most important task.
- Prepare enough, time is precious, if you want to make it in the Hackathon project, do some preparations, it’s not cheat!