Intro
The meta-erlang on cloud II, build your own application post has shown the basic idea for how would be possible (and in fact it is possible) to build a custom Linux distribution and run it on Graviton AWS instances.
Now, it's time to refactor out the previous result. Removing what are not necessary in order to create a slim meta-erlang cloud ready image.
So, this post is more like a tutorial to reach two main targets:
- Explore a demo Yocto layer which shows the basic pieces to run a distro on Graviton
- Add livebook to that image and try it on cloud. Why not trying some GPU processing :)
meta-axon cloud aware
meta-axon is my playground layer focused on demonstrate how Erlang and Elixir applications could be integrated with Yocto ecosystem. Until now, meta-axon has been tailored to run on embedded devices (like raspberry pi). But, I'm also added, along side with the former axon-slim, a new distro called axon. The axon distro is focused on cloud, for the purpose of this tutorial AWS cloud.
If you are new to the Yocto Project; remember some basic concepts here.
I also introduced a new image called axon-image-cloud with all the pieces needed to run it on Graviton instances. This image relies on systemd plus cloud-init scripts.
It is important to mention that to run images on Graviton it is necessary to add two layer dependencies:
- meta-arm, implementing a generic ARM 64 bits machine
- meta-virtualization, bringing cloud-init recipe
The rest of the needed components are all provided by standard Yocto layers (like: poky and meta-openembedded).
Build instructions
We want fast feedback and test the build results quickly. So here is the procedure that will build an distro linux image with Erlang, Elixir and livebook:
I'm assuming a working Linux environment in order to run the below commands.
Clone the meta-axon layer:
git clone https://github.com/meta-erlang/meta-axon.git
cd meta-axonInstall the kas tool to setup a bitbake based project.
git clone https://github.com/siemens/kas
cd kas
pip3 install .Call kas tool to build everything:
kas build kas/machines/meta-erlang-graviton2-livebook.yamlnoteThe final image has been created in the deploy folder: tmp/deploy/images/generic-arm64.
Run the script create-ami.sh to convert and upload a image made with Yocto into AWS AMI. Use your s3 bucket and specific how much extra space you want:
../scripts/create-ami.sh <add your s3 bucket here> 8
The script create_ami.sh may need a fix in order to find the correct place for the qemu-img tool. Would be nice to have some sort of Yocto bbclass to automatically convert and push the AMI image to AWS. But it is not the case yet.
Launching EC2 instances
After uploading the image, it's time to lunch an EC2 instance like the following:
Remember to use a Free Trial available instance type. When I wrote this post, the eligible tier was the t4g.small.
And, finally here is it running:
Accessing the instance via ssh and login with the user axon should work as expected. After all it is a normal Linux distro.
Results
The following two screenshots show the livebook instance up and running on Graviton EC2 instance. It is pretty cool.
Just showing the top command to check all the running process:
And here is the livebook system dashboard to inspect some usage statistics, and system information:
As you can see, it is running on ARM64, with JIT enable.
Conclusion
Now we are converging to create the basic building blocks for creating custom Linux distribution based on Yocto Project that runs on Cloud.
That could be the perfect case for projects that need to take control on every detail of the software stack. From linx kernel configuration, building flags, software configuration flags, anything. It's really flexible.
And for projects that relies on Erlang and Elixir, it is also amazing bringing it to Cloud ecosystem.
The first blog part meta-erlang on cloud I: EWAOL based has introduced what is possible to build. This blog post is more related to explore the introduced idea into something feasible and aligned with meta-erlang purposes.