Building a Vault Enterprise Node for testing with Vagrant

vagrant+vault_blog.png

Over three years ago, I started to play with Vagrant and what I could build when testing Oracle GoldenGate. Since them Vagrant has become a tool in my toolbox for quickly building Virtual Box (VBox) machines for testing out different tools and utilities. Recently one of those tools that has caught my attention is the HashiCorp Vault (Vault) product.

Vault is a Privileged Access Management (PAM) software that is used to keep “secrets”. You can secure, store and tightly control access to tokens, passwords, certificates, and a whole host of other sensitive data. Then your stored “secret” can be accessed by UI, CLI, or HTTP API.

In this post, I’ll show you some basics of how I built a HashiCorp Vault Enterprise Node for testing with Oracle products. I’m using HashiCorp Vagrant, HashiCorp Vault Enterprise, and Oracle VirtualBox.

Environment:

Oracle VirtualBox (VBox) 6.1.12
HashiCorp Vagrant (Vagrant) 2.2.9
HashiCorp Vault Enterprise (Vault) 1.5.3

Sizing:

1GB of memory
2 virtual cpus

Note: HashiCorp Vault Enterprise is a licensed product with enterprise features. For just building at testing node, using the open source version of Vault is acceptable.

The Beginnings:

With any virtual machine that you would build with Vagrant, requires a Vagrantfile. The Vagrantfile is the control file for building the virtual machine. The Vagrantfile is written in Ruby and provides the details on the structure of the virtual machine – items like OS, network, memory, CPU, etc.. Additionally, the file uses “provisioners” that allow you to tell the VM what to execute upon building of the machine. In the example below, I’m using the shell provisioner to setup environment variables that will be use for the install.

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

BOX_IMAGE = "ol7-latest"
BOX_URL = "https://yum.oracle.com/boxes/oraclelinux/latest/ol7-latest.box"

# Name of the VM and the hostname inside
NAME1 = "Vault"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  #config.vm.network "private_network", type: "dhcp"
  config.vm.hostname = "VaultEnterprise"

    config.vm.define "vault" do |vault|
      vault.vm.network "private_network", :auto_network => true
      vault.vm.box = BOX_IMAGE
      vault.vm.box_url = BOX_URL
      vault.vm.hostname = NAME1

    # VM Customization
    vault.vm.provider "virtualbox" do |v|
        v.customize [ "modifyvm", :id, "--memory", 1024 ]
        v.customize [ "modifyvm", :id, "--cpus", 2 ]
        v.customize [ "modifyvm", :id, "--paravirtprovider", "kvm" ]
        v.customize [ "modifyvm", :id, "--name", NAME1 ]

      vault.vm.network "forwarded_port", guest: 8200, host: 8200
      #Used when using the Raft protocol
      #vault.vm.network "forwarded_port", guest: 8201, host: 8201

      vault.vm.synced_folder "/Users/bocurtis/Downloads", "/Test_Software"

      vault.vm.provision "provision_vault", type: "shell", path: "scripts/install.sh", env:
      {
        "CLIENT_SHIPHOME_19C"    => "instantclient-basic-linux.x64-19.8.0.0.0dbru.zip",
        "HASHICORP_BASE"         => "/opt/app/hashicorp",
        "ORACLE_CLIENT_HOME"     => "/usr/lib/oracle/19.8/client64",
        "HASHICORP_BIN"          => "/opt/app/hashicorp/bin",
        "HASHICORP_CONFIG"       => "/opt/app/hashicorp/config",
        "HASHICORP_DATA"         => "/opt/app/hashicorp/data",
        "HASHICORP_PLUGIN"       => "/opt/app/hashicorp/plugin",
        "VAULT_ORA_PLUGIN"       => "vault-plugin-database-oracle_0.2.1_linux_amd64.zip",
        #change after 30 days-9/4/2020
        "VAULT_SOFTWARE"         => "vault_1.5.3+ent_linux_amd64.zip",
        #"VAULT_SOFTWARE"         => "vault_1.5.2_linux_amd64.zip",
        "LD_LIBRARY_PATH"        => "/usr/lib/oracle/19.8/client64/lib"
      }
    end
  end
end

Installation:

Once Vagrant reads the Vagrantfile, the VM will start to be built. This typically starts with downloading a pre-built vagrant box. Once the box is installed and brought online then the “provisioners” will install the required software (rpms, binaries, etc …). In building this HashiCorp Vault node, installing Vault is pretty simple (as you will see), but the extra layer that has to be installed is Oracle Instant Client since I’m wanting to use the Oracle Vault plugin.

Oracle Instant Client:

There are multiple ways of installing Oracle Client and I typically do a custom install of the client; however, for the Oracle Vault plug-in to work – the instant client has to be installed via yum. If not, then you may have to recompile the plug-in to ensure it works. During this installation process, I figured out that you have to install the developer tools, tools and sqlplus as well. This allows you to test different aspects of connecting to the databases that will be used within Vault.

echo
echo '###################################'
echo  'Installing Oracle Instanct Client 19.3.0.0'
echo '###################################'
echo
echo

yum -y -q install oracle-release-el7
yum -y install oracle-instantclient19.8-basic.x86_64
yum -y install oracle-instantclient19.8-devel.x86_64
yum -y install oracle-instantclient19.8-tools.x86_64
yum -y install oracle-instantclient19.8-sqlplus.x86_64

Adding a Vault User:

When you start working with Vault initial (at least I did), Vault is installed under root. Using the root user is not a good practice to get into and most organizations will not allow you to do that anyways. A good thing to do would be to create a vault user. This user will be used to run the Vault product.

echo
echo '###################################'
echo  'Adding Vault User'
echo '###################################'
echo
echo

#Adding Vault User
useradd -d /home/vault -s /bin/bash vault
echo "vault" | passwd vault --stdin

Creating Directories:

The next thing that should be done is creating the directories where the Vault software will be installed. Notice that the directory locations were set in the “provisioner” in the Vagrantfile. The shell script is just calling these environment variables when the commands are executed.

echo
echo '###################################'
echo  'Creating HashiCorp Directories'
echo '###################################'
echo
echo

mkdir -pv $HASHICORP_BASE && \
mkdir -pv $HASHICORP_BIN && \
mkdir -pv $HASHICORP_CONFIG && \
mkdir -pv $HASHICORP_DATA && \
mkdir -pv $HASHICORP_PLUGIN

#set permissions on Vault directories
chown vault:vault -R $HASHICORP_BASE
chown vault:vault -R $HASHICORP_BIN
chown vault:vault -R $HASHICORP_CONFIG
chown vault:vault -R $HASHICORP_DATA
chown vault:vault -R $HASHICORP_PLUGIN

Installing Vault and Oracle Plug-In:

With everything set, the installation process can now begin to install Vault and the plug-in to use Vault with Oracle. At this point, notice that the Vault software is being unzipped along with the plug-in. Then runs a setcap command to give the Vault executable the ability to use the mlock sys call without being root. Next (not shown), is echo commands for building out vault.hcl file and the system services file. The vault.hcl file is used to configure Vault and how it should start. The system services file is just setting Vault up to run as a background service which can be controlled by systemctl. Then configure Vault to start on reboots with the systemctl commands before sleeping for 5 seconds.

At this point, the installation process is initializing Vault and capturing the needed encryption keys and root token to unseal the Vault. These values are saved into a plain text file on the VM for later retrieval, but are used in a more dynamic way when unsealing the Vault. Next, the vault user has their .bash_profile configured. This allows for the environment to be set correctly when accessing the Vault user to work with Vault. Lastly, the Vault executable is ran to unseal the Vault and to login. The keys that were captured upon initialization are used for this process.

echo
echo '###################################'
echo  'Installing HashiCorp Vault Software'
echo '###################################'
echo
echo

#Installing Vault and Plugin
unzip -q /vagrant/hashicorp_software/${VAULT_SOFTWARE} -d ${HASHICORP_BIN}
unzip -q /vagrant/plugins/${VAULT_ORA_PLUGIN} -d ${HASHICORP_PLUGIN}

setcap cap_ipc_lock=+ep $(readlink -f /opt/app/hashicorp/bin/vault)

#Setup Vault Config file (vault.hcl)

#Set Vault up as systemd service

systemctl start vault
systemctl enable vault
systemctl status vault

sleep 5

export PATH=${HASHICORP_BIN}:$PATH
export VAULT_ADDR=http://localhost:8200
vault operator init -recovery-shares=1 -recovery-threshold=1 > /home/vault/init.txt 2>&1
export UNSEAL_KEY_1=`cat /home/vault/init.txt | sed -n -e '/^Unseal Key 1/ s/.*\: *//p'`
export UNSEAL_KEY_2=`cat /home/vault/init.txt | sed -n -e '/^Unseal Key 2/ s/.*\: *//p'`
export UNSEAL_KEY_3=`cat /home/vault/init.txt | sed -n -e '/^Unseal Key 3/ s/.*\: *//p'`
export VAULT_TOKEN=`cat /home/vault/init.txt | sed -n -e '/^Initial Root Token/ s/.*\: *//p'`

#Setting Environment Variables
echo "export PATH=${HASHICORP_BIN}:$PATH" >> /home/vault/.bash_profile
echo "export VAULT_ADDR=http://localhost:8200" >> /home/vault/.bash_profile
echo "export VAULT_API_ADDR=http://localhost:8200" >> /home/vault/.bash_profile
echo "export VAULT_TOKEN=${VAULT_TOKEN}" >> /home/vault/.bash_profile
echo "export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}" >> /home/vault/.bash_profile
echo "export HASHICORP_HOME=${HASHICORP_BASE}" >> /home/vault/.bash_profile

#Unsealing and logging in to Vault
vault operator unseal $UNSEAL_KEY_1
vault operator unseal $UNSEAL_KEY_2
vault operator unseal $UNSEAL_KEY_3
vault login $VAULT_TOKEN

Installing Plug-in

With Vault installed and unsealed, the next thing that has to be done is write your license key to so Vault features are all available and to install the Vault Oracle Plug-in. For writing the license key, a simple vault write command is used and passed a string of encrypted characters. After the license key has been installed, then the Oracle Plug-In can be installed. The Plug-in installation is requires you to capture the SHA256 key. With the SHA256 key in hand, enable the database secret engine before writing the plug-in to Vault.

#Write License Key
#comment out after 30 days
vault write sys/license text=02MV4UU43BK.........2PI

#Get SHA256 info
shasum -a 256 /opt/app/hashicorp/plugin/vault-plugin-database-oracle | sed 's/\s.*$//' > /opt/app/hashicorp/plugin/oracle-plugin.sha256
chown vault:vault ${HASHICORP_PLUGIN}/oracle-plugin.sha256

#Enable Database Key Vault
vault secrets enable database

#Write plugin to vault
vault write sys/plugins/catalog/database/oracle-database sha256=$(cat /opt/app/hashicorp/plugin/oracle-plugin.sha256) command=vault-plugin-database-oracle

Up and Running:

With everything installed, I now have a fulling functioning HashiCorp Vault node running. The only thing that needs to be done now is access the node by using the root token that was identified in the $VAULT_TOKEN environment variable.

After logging in as the root user for Vault, you will land on the “Secrets” page. This page provides a summary of what “Secret Engines” are currently installed and provided to use within Vault.

Summary:

When setting up a development environment, it is good to have a “secrets” tool that can be used to control access to “secrets” as needed. There is a lot that you can do with Vault and the setup is pretty easy. Coupling this with Vagrant makes for a quick deployment within a testing environment. When wanting to expand out to a larger environment there is much more you can do, but that is not the topic for discussion at this time. I would encourage you to take a look at HashiCorp Vault and how it can be used to secure your on-premises environments along with your cloud environments.

Enjoy!!!

Please follow and like: