Our (Almost) Zero Touch Deployment
·6 min read
Like many other high-growth companies, we did almost all of our onboarding process by hand when we first started out. As we scaled, however, we quickly ran into challenges and needed to automate as many tasks as possible. We estimated that each new hire took about five hours to be fully set up for their first day. Not only is this not scalable, this didn’t follow our ideals of automating as much as we can.
It’s exciting to work at a rapidly growing company, but with rapid growth comes the logistical challenge of onboarding at scale. We recognized that we needed to streamline our processes before the large hiring waves came in.
So we automated (almost) everything.
Since this time last year, our monthly new hire count has doubled but our time to onboard has been cut down to about 2.5 hours (this even includes time for system updates to apply).
Why is this a big deal?
We completed all three initiatives in less than one year, where normally a single initiative would take more than a year to be researched, tested, rolled out, and considered “done”.
I’ll walk through each of these three initiatives, but first I’ll give a bit of background on our environment setup and how it plays a key role in our ongoing success.
Our environment setup
We are 100% Mac computers
Early on, our leadership team made the decision to dedicate us to one operating system (OS) and manufacturer. This was done for two reasons:
- IT only had to worry about one set of peripherals, cables, and hardware when ordering and servicing.
- IT and Security only had to keep track of securing a single OS.
A bonus side effect of having a single OS was that we could focus on implementing a mobile device management (MDM) that fit our exact need instead of trying to find an MDM that worked across different OSs.
We use hosted services where we can
At Signal Sciences we’re all about optimizing in the right places. We offloaded a lot of overhead from our SRE and IT teams by deciding that we should use Software as a Service (SaaS) hosted services. SaaS hosted can potentially cost more than hosting yourself, but the returns on not spending time setting up, updating and securing servers, or performing backups pay off quickly and allows your team to be forward thinking.
How we did it
Intel gathering
Before entering this role, some of the planning work was already completed; we already knew what our device footprint was based on department and role. The next step was to meet with the hiring managers and gather which applications, email groups, slack channels, etc. each of their new hires would need for their first day.
We now had enough metadata to know:
- Who should have access to specific applications
- What their level of access should be
This is important in initiatives 2 & 3.
Initiative 1 - You got JAMF’D (automating computer setups)
We tackled automating the setup of our user systems first. Up to a certain point, every computer was getting the exact same configuration. The devices get a password policy, FileVault was enabled, the OS was getting all the latest updates, base software was installed, and more. By getting the device setups automated, we freed up more time for us to research and plan our next steps.
To achieve this we leveraged JAMF and Apple’s Device Enrollment Program (DEP). We skipped the out-of-the-box experience, updated MacOS, automated enabling FileVault and our other security settings. With JAMF, a single person can set up five devices at the same time and the only reason they could not do more is because there’s not enough room on their desk.
Initiative 2 - One ring to rule them all (getting rid of the janitor’s keychain)
We knew from the start that we wanted Okta as our Single Sign-On (SSO) provider. With Okta we could protect all of our applications with the same password policy, always enforce Multi-factor Authentication (MFA), and keep track of user access privileges.
We automated access into everything from video conferencing apps to AWS access. The AWS access was a big game changer for us at the user and access key management level. By delegating authentication to Okta, we never have to manually deal with user provisioning again. All accounts and API keys are automatically provisioned and expire after usage.
The user metadata that we normalized earlier is consumed in Okta to automatically assign users to their relevant groups. These groups were then assigned to one or more applications or left alone for our next initiative.
Alongside our SSO deployment we also deployed an LDAP server, which gave us the following abilities:
- RADIUS Secured Wi-Fi
- Users and Groups are syncing from Okta
- One less password to manage (authentication delegated to Okta)
- One less account to disable
Initiative 3 - Trust but verify
We like the Least Privilege model, but traditional VPNs do not make this very easy to accomplish, nor are they easy to set up quickly and consistently. Normally when you get access to a VPN you get access to everything behind it. You could set up a bunch of rules and conditions as to who could get what but then you’re stuck managing it.
We were already using Duo as our MFA provider and so we looked at Duo Beyond (similar to Google’s BeyondCorp Zero Trust). By leveraging Duo Beyond to allow users access to specific web applications and drive those permissions off of the groups that were populated in our LDAP via Okta. Duo also validates that employees are on a company-provided device, and checks for other aspects of the device’s health, before allowing the employee to connect to an internal app.
Our Ops team utilizes Terraform for our Infrastructure as Code, so we built a module that would stand up the entire stack in AWS for Duo Beyond to work in siloed deployments. Now we don’t need to setup a VPN server for a single application––instead we could deploy a stack into the VPC with a unique DNS namespace and be ready to provide access to users in less than 20 minutes.
What’s next?
So far we’ve saved time by automating the following:
- Provisioning and Deprovisioning of accounts
- Ensuring only necessary access is granted based on the Least Privilege model
- Leveraging Terraform to automatically and consistently set up infrastructure
Our next focus is having our user systems configure the role-specific applications that get deployed to their laptops based on who is assigned the system. This will cut down the training and onboarding time that users spend with their hiring managers.