Category Archives: linux

h5py python causes TensorFlow installation error for Jetson Nano

Posted on by
Reply

After forcing the Jetson OS (based on the ubuntu 18.04) to have Python 3.8 running. After having the JetPack 4.6.3 installed in the Jetson Nano Jetson OS.

When attempting to install TensorFlow 2+ manually in the Jetson Nano based on Official TensorFlow for Jetson Nano! – Jetson & Embedded Systems / Jetson Nano – NVIDIA Developer Forums

Running the installation in Jetson Nano

Or running the pip3 install command:

sudo pip3 install --pre --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v46 tensorflow

After a long time installing all the dependency for TensorFlow. A dreaded error message will occur.

ERROR: Failed to build installable wheels for some pyproject.toml based projects (h5py)

ERROR: Failed to build installable wheels for some pyproject.toml based projects (h5py)

To understand the error, scroll up to read what are the error.

Continue reading →

Adding compatible SSD into seeed reComputer J1020

Posted on by
Reply

Adding SSD into will requires M.2 nvme into the reComputer J1020, this post is based on Memory Expansion | Seeed Studio Wiki

The documentation is a bit vague on the NVMe SSD to be used other than very basic instructions. However, in my attempt it is smooth.

The SSD that was chosen as the expansion storage is the Kingston NV2 PCIe NVMe M.2 500GB. The J1020 requires the SSD to be M key.

Kingston NV2 PCIe NVMe M.2 500GB, M key
Continue reading →

Using AWS metadata to manage and automate tasks in AWS EC2 or AWS Lightsail

Posted on by
Reply

This post is based on reference to Use instance metadata to manage your EC2 instance – Amazon Elastic Compute Cloud

Metadata is a powerful tool for AWS users. It allows users to make query of data describing EC2 instances, and making a self reference API call.

By default any linux AMI will have curl build in, hence using of metadata will be simplified.

One of the best test case is automating input into shell script that will requires a lot of user prompt is automating or at least make configuration of setting up openvpn using AWS Lightsail easier.

!#/bin/bash
sudo chmod 777 ./openvpn-install.sh
sudo ./openvpn-install.sh << INPUT


y
1
1
11
n
n

client
1

INPUT
sudo cp /root/client.ovpn /home/ubuntu
sudo chmod 777 /home/ubuntu/client.ovpn

Above shell script will cause the openvpn-install.sh to fail in AWS, as the script does not provide public and local ip of the instance.

Below script are including the AWS Metadata

!#/bin/bash
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
PUBLICIP=`curl -X GET "http://169.254.169.254/latest/meta-data/public-ipv4" -H "X-aws-ec2-metadata-token: $TOKEN"`
LOCALIP=`curl -X GET "http://169.254.169.254/latest/meta-data/local-ipv4" -H "X-aws-ec2-metadata-token: $TOKEN"`
sudo chmod 777 ./openvpn-install.sh
sudo ./openvpn-install.sh << INPUT
$LOCALIP
$PUBLICIP
y
1
1
11
n
n

client
1

INPUT
sudo cp /root/client.ovpn /home/ubuntu
sudo chmod 777 /home/ubuntu/client.ovpn

By adding the metadata the ovpn file will be populated with correct IP.

AWS Metadata allow automation to be made simpler by running scripts that requires self-reference metadata to configure newly booted up EC2 instance(s).

Playing Terraform for AWS

Posted on by
Reply

This post is based on terraform tutorial Build infrastructure | Terraform | HashiCorp Developer

Make sure AWSCli being installed and configured correctly (aws configure). Make sure IAM user are configured with AWS role, AWS Access key ID and AWS Secret Access Key.

Use the ami catalog, to determine the ami ID, then after make necessary changes run the init

Result of terraform init
terraform plan will show configuration to be applied once init are successful
AWS EC2 creation failed

To solve this VPC and subnet needs to be created first.

Creation of VPC completed, copy the DMZ subnet ID

Make small change to the EC2 instance of terraform file.

Add DMZ subnet to the terraform file

Rerun the terraform init, terraform plan, terraform validate before rerunning terraform apply, then type yes.

EC2 provisioning Completed
Cleaning up, terraform destroy

To clean up the experiment to avoid paying more, start by destroying the instance then the VPC.

Terraform code is available at https://bitbucket.org/KarMeng/terraform_aws

Experimenting with Terraform

Posted on by
Reply

All the experience in this post is based on Install Terraform | Terraform | HashiCorp Developer

After terraform init, terraform apply needs confirmation of “yes” before applying into environment

Feels like running ansible but simpler, as the tutorial of quickstart runs on docker engine.

After applying terraform, confirm the nginx is running

Further confirmation running browser to browse the site.

The terraform tutorial are running nginx at port 8000 of your host machine.
If you preferred curl

How to use docker compose to setup AWStats

Posted on by
Reply

Have added changes to incorporate both generating AWStats logs and starting up AWStats service in a single docker compose file at KarMeng / docker_awstats — Bitbucket

Sample docker compose for AWStats

This is an easy and simple example that beginners can use to generate web statistics using AWStats.

Required softwares:
Hashicorp Vagrant 2.4.1
Oracle VirtualBox 7.0.14

Continue reading →

ElasticSearch 8.12 docker compose do not work

Posted on by
Reply
Error message exited 137

The first error that will be face on get go is the error “kibana Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap” and exiting error “dependency failed to start: container docker-es03-1 exited (137)”.

Searching mentioned error on google or the internet will yield result that advice swap memory and memory limit hit.

Continue reading →

ElasticSearch 8.12 kibana cluster using vagrant and docker compose

Posted on by
Reply

Pre-requisite:
– VirtualBox 7.0.14
– Vagrant 2.4.1
– Windows 10 or better OS
– 16GB RAM (10GB RAM are required for creating ElasticSearch with kibana; 1GB and 2 x ElasticSearch node; 4GB each, rest of the RAM for VM host OS)

Overview:
There are 2 layers of virtualization, first the Virtual Box, then the docker engine running in the Virtual Box VM running on Ubuntu 20.04 focal.

Orchestration used in the host OS level; Windows 10 are the hashicorp vagrant. The vagrant is used to configure the VM Ubuntu OS to be configured to run properly configured docker and Ubuntu 20.04.

Then docker compose v2 are used to create the ElasticSearch 8.12 cluster or stack.

The downside of this example, vagrant up needs to be run initially to configure the VM Ubuntu 20.04 OS. I have yet to discover if Vagrant has the ability to bootstrap grub and configuring the sysctl to allow the docker engine to run properly with the ElasticSearch 8.12 stack.

Continue reading →

How to use Hashicorp Vagrant to quickstart GitLab docker compose sample

Posted on by
Reply

The code of the project is available at chowkarmeng/vagrant_gitlab (github.com), the docker compose is based on the sample provided in GitLab Docker images | GitLab

The improvement done was to change the folder sync for virtual box into docker volumes.

First, git clone the repository https://github.com/chowkarmeng/vagrant_gitlab.git

Fire up the quickstart by running “vagrant up” in the localdev

The process will take hours depending on the speed of your computer and speed of your internet connection.

Continue reading →