Playing Terraform for AWS

This post is based on terraform tutorial Build infrastructure | Terraform | HashiCorp Developer

Make sure AWSCli being installed and configured correctly (aws configure). Make sure IAM user are configured with AWS role, AWS Access key ID and AWS Secret Access Key.

Use the ami catalog, to determine the ami ID, then after make necessary changes run the init

Result of terraform init
terraform plan will show configuration to be applied once init are successful
AWS EC2 creation failed

To solve this VPC and subnet needs to be created first.

Creation of VPC completed, copy the DMZ subnet ID

Make small change to the EC2 instance of terraform file.

Add DMZ subnet to the terraform file

Rerun the terraform init, terraform plan, terraform validate before rerunning terraform apply, then type yes.

EC2 provisioning Completed
Cleaning up, terraform destroy

To clean up the experiment to avoid paying more, start by destroying the instance then the VPC.

Terraform code is available at https://bitbucket.org/KarMeng/terraform_aws

Experimenting with Terraform

All the experience in this post is based on Install Terraform | Terraform | HashiCorp Developer

After terraform init, terraform apply needs confirmation of “yes” before applying into environment

Feels like running ansible but simpler, as the tutorial of quickstart runs on docker engine.

After applying terraform, confirm the nginx is running

Further confirmation running browser to browse the site.

The terraform tutorial are running nginx at port 8000 of your host machine.
If you preferred curl

How to use docker compose to setup AWStats

Have added changes to incorporate both generating AWStats logs and starting up AWStats service in a single docker compose file at KarMeng / docker_awstats — Bitbucket

Sample docker compose for AWStats

This is an easy and simple example that beginners can use to generate web statistics using AWStats.

Required softwares:
Hashicorp Vagrant 2.4.1
Oracle VirtualBox 7.0.14

Continue reading

ElasticSearch 8.12 docker compose do not work

Error message exited 137

The first error that will be face on get go is the error “kibana Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap” and exiting error “dependency failed to start: container docker-es03-1 exited (137)”.

Searching mentioned error on google or the internet will yield result that advice swap memory and memory limit hit.

Continue reading

ElasticSearch 8.12 kibana cluster using vagrant and docker compose

Pre-requisite:
– VirtualBox 7.0.14
– Vagrant 2.4.1
– Windows 10 or better OS
– 16GB RAM (10GB RAM are required for creating ElasticSearch with kibana; 1GB and 2 x ElasticSearch node; 4GB each, rest of the RAM for VM host OS)

Overview:
There are 2 layers of virtualization, first the Virtual Box, then the docker engine running in the Virtual Box VM running on Ubuntu 20.04 focal.

Orchestration used in the host OS level; Windows 10 are the hashicorp vagrant. The vagrant is used to configure the VM Ubuntu OS to be configured to run properly configured docker and Ubuntu 20.04.

Then docker compose v2 are used to create the ElasticSearch 8.12 cluster or stack.

The downside of this example, vagrant up needs to be run initially to configure the VM Ubuntu 20.04 OS. I have yet to discover if Vagrant has the ability to bootstrap grub and configuring the sysctl to allow the docker engine to run properly with the ElasticSearch 8.12 stack.

Continue reading

How to use Hashicorp Vagrant to quickstart GitLab docker compose sample

The code of the project is available at chowkarmeng/vagrant_gitlab (github.com), the docker compose is based on the sample provided in GitLab Docker images | GitLab

The improvement done was to change the folder sync for virtual box into docker volumes.

First, git clone the repository https://github.com/chowkarmeng/vagrant_gitlab.git

Fire up the quickstart by running “vagrant up” in the localdev

The process will take hours depending on the speed of your computer and speed of your internet connection.

Continue reading

Vagrant to orchestrate ubuntu in VirtualBox installing boto3 and Ansible

At the time of this post, the compatibility matrix of vagrant and VirtualBox is as follows:

Vagrant versionVirtualBox version
2.3.77.0.10
7.0.12
Vagrant and Virtualbox compatibility matrix

Unfortunately, Vagrant 2.4.0 does not work well with VirtualBox 7.0

This post was created using Vagrant 2.3.7 and VirtualBox 7.0.10

To make Vagrant possible, after installing the Vagrant from hashicorp webpage, a Vagrant file needs to be created. The most basic file that needs to exist in your Vagrant to work is a folder with Vagrantfile

Additional post start up scripts to complete the installations that are used in this example are setup.sh, install_ansible.sh, install_boto3.sh and install_python3.sh

Continue reading

Setting up OpenVPN using Amazon Lightsail

Pre-requisites:
1. Download “openvpn-install.sh” GitHub – angristan/openvpn-install: Set up your own OpenVPN server on Debian, Ubuntu, Fedora, CentOS or Arch Linux.
2. Have Amazon Lightsail activated with quota in AWS account.
* Credits to cyberciti for instructions and the scripts introduction Ubuntu 20.04 LTS Set Up OpenVPN Server In 5 Minutes – nixCraft (cyberciti.biz)
3. SSH keypair are created and added into your AWS account.
4. Make sure OpenVPN client is install on your computer OpenVPN Connect – Client Software For Windows | OpenVPN

Steps:
Creating a Ubuntu AWS instance
1. Click on “Create Instance”

2. Select OS only, and choose your favorite Linux distro, in my case Ubuntu were chosen.
At the time of this blog post created, the openvpn install script works on both ubuntu 22.04 and 20.04

Continue reading

how to install mysql proxy quick and easy

In the linux ubuntu or redhat/centos (yum) just use the default package manager. For my post I am using ubuntu as an example to illustrate my point.

sudo apt-get install mysql-proxy

You should be getting the following confirmation

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
ttf-dejavu-extra libevent-extra-2.0-5 libmysql++3 libdbi1 libapr1 librrd4 libcairo2 libmysqlclient-dev libaprutil1-ldap libthai-data libreadline6-dev libpcrecpp0 libsvn1 libdatrie1
fontconfig libtinfo-dev libpixman-1-0 libevent-openssl-2.0-5 libaprutil1-dbd-sqlite3 libonig2 libthai0 libneon27-gnutls zlib1g-dev libevent-pthreads-2.0-5 libzzip-0-13 ttf-dejavu
libdb4.8 libpq5 libpcre3-dev libpango1.0-0 libxcb-render0 libxcb-shm0 libaprutil1 libevent-core-2.0-5 libfcgi0ldbl libreadline-dev

However, to run the mysql-proxy there is a few tweak needed, assuming that you are not using mysql-proxy beyond just a proxy.

To start up mysql-proxy, you may issue the command such as this :
sudo mysql-proxy --defaults-file=/etc/mysql/mysql_proxy.cnf &

The configuration of mysql_proxy.cnf :

log-file = /opt/apps/logs/mysql-proxy/mysql-proxy.log
log-level = debug
proxy-backend-addresses = 10.161.89.64:3306
admin-username = root
admin-password = for_my_eyes_only

If the mysql-proxy fails to start as a daemon, it is best to check the logs at /opt/apps/logs/mysql-proxy/mysql-proxy.log :

root@ckm-myprox:/opt/apps/logs/mysql-proxy$ sudo tail -f mysql-proxy.log
2014-06-11 20:58:27: (message) mysql-proxy 0.8.1 started
2014-06-11 20:58:27: (debug) max open file-descriptors = 1024
2014-06-11 20:58:27: (critical) admin-plugin.c:579: --admin-lua-script needs to be set, /lib/mysql-proxy/lua/admin.lua may be a good value
2014-06-11 20:58:27: (critical) mainloop.c:267: applying config of plugin admin failed
2014-06-11 20:58:27: (critical) mysql-proxy-cli.c:596: Failure from chassis_mainloop. Shutting down.
2014-06-11 20:58:27: (message) Initiating shutdown, requested from mysql-proxy-cli.c:597
2014-06-11 20:58:27: (message) shutting down normally, exit code is: 1

The last line of the log, just confirmed the mysql-proxy were failed to start due admin lua were not set. To skip the admin lua function(assuming that it will not be used). Start the mysql with :
sudo mysql-proxy --defaults-file=/etc/mysql/mysql_proxy.cnf --plugins=proxy &

Take note that my post on installing mysql-proxy, the version is 0.8.4 and the stock from ubuntu repository is 0.8.1 .