Rob Ward's Log

2014-10-30

Making progress slowly, not getting much time at the moment, I do however commute a long way so have managed to spend a bit of time on my less busy trains getting a few bits implemented.

So far I have added the Linux stable repo into my project as a submodule. I am using the upstream version from kernel.org as I don't currently need to modify any of the kernels source. If that changes I will switch to using my own version hosted on GitHub.

As well as adding the Linux kernel source I have also added a first version of a makefile. Currently it is slightly messy and only defines a few variables and a couple of targets. The main target is for building the Linux kernel.

Currently the way I build the kernel is hacky but it works as a first stage. The makefile itself is in the git repo already but the following in the set of commands that are in the make target:



I have chosen to use the default x86_64_defconfig, this keeps the current implementation very simple. I will however need to implement the use of custom kernel configs and the ability to build different architecture in the not too distant future.

What the current build target does do however is build the kernel modules and install them in a location that can then be included in the initramfs. I have also put some sed commands in place that setup the source location for the initramfs and the UID and GID. The sed commands are rather evil but they work for now.

The CONFIG_INITRAMFS_SOURCE is used to give a list of locations of files to include in the initramfs. This can either be a config file or a directory. If it is a directory then the contents are included in the initramfs. In my current implementation I pass both a config file and a directory. The directory that I pass is the INITRAMFS_DIR, this is where the kernel modules are installed after building and where I will install any of the other utilities that I need in the initramfs going forward. The config file is a plain text file that is used to specify directories or device nodes that should be created. The current config file I am using is very simple and creates the directories /dev, /proc, /sys and the device node /dev/console. My current config is in the git repo but looks like this:




The CONFIG_INITRAMFS_ROOT_{UID/GID} are used to tell the kernels build system to map the files user/group permissions from the given UID/GID to be owned by the root user.

You will notice looking at the build commands above that I am calling make with --jobs=4 in order to build multi-threaded. I chose 4 simply because I was on my laptop with 4 CPU's. A future enhancement will make the number of jobs dynamic depending on the number of cores the machine building has, well unless I add distcc into the mix.

So where does that leave me at the minute? Well if I run the make command a kernel is successfully built. The end result of the build is:




This bzImage file is the kernel + initramfs image that we can run.

To run the image I am currently using qemu, the command I am using to run it is:




When run the kernel boots up until the point where it panics and explodes horribly:




Generally when you boot a Linux kernel it will try and run the application /init which is used to setup and configure userspace and run every other application on your system. The build I ran above didn't have an init so once the kernel finishes booting it has nothing to call and dies horribly.

As such my next step was to create a minimal init application to include in the initramfs. The simplest way of creating a init application is a statically linked C application. I have created this and placed in another git repo called microinit[0] that I have added to GitHub. I have then added this as a submodule and added the microinit target in the makefile.

At this point I now have a kernel+initramfs that boots and runs, admittedly all it does at the moment is sit and say "Microinit is running" once a second, but small steps.

My next stage will be to get an actual command line up and running. I have usually used Busybox[1], however for what I am doing as space/speed isn't really the point I could use the full utilities as ship with most distros. The other choice is the use a newer package called Toybox[2], I have never used this before so I may have to give it ago, I know it isn't finished but it may do what I need at the moment.

I also have to start thinking about toolchains and how to manage them. Currently I think I will probably build a Toolchain(or a few) and store them as binaries rather than building a toolchain every time.

One other think I am considering doing is making all of this build in a fakechroot environment, I have used this a few times for compiling software and it does make cross compiling easier for certain uncooperative packages, I haven't built a full system using it though. Again this would be something new to try so I think I will have ago and see what happens. It does mean that I will need to build a number of the host tools, or take copies of them into the fakechroot along with the compiler that I am using to cross compile with. This adds complexity but would make the system more self contained.

[0] - https://github.com/rob-ward/microinit
[1] - http://www.busybox.net/
[2] - http://landley.net/toybox/



2014-10-28

So, the first stage for me to build a custom Linux distro is to sort out repos for the build system and packages I want to build.

I previously decided I was going to use github to host this work, at least initially. As such my plan is to create a single repo to store my build system. I will also have a separate repo for storing each of the packages I am going to build, at least I will for any that are not already available in a git repo or where I need the ability to modify them.

I generally use the git-repo[0] tool developed by Google for the android project when I want to manage multiple git repos. Git-repo works very well and gives you a lot of control, especially when sharing code between multiple teams.

I have however decided that I am going to use git submodules[1] instead, I have used these in the past and they should do what I need at the moment, I can always change later easily enough if I need to.

So to start with I will be creating a repo that will manage the submodules and also hold the top level build system and documentation. Below this I will place all the source that I am building as separate repos.

In order to create a repo I need a name and custom-linux-distro just doesn't quite work for me. As such I have decided, at least for now, to call it intrepid-Linux.

So that is it, I have created the repo[2] and all I have to do is start adding code and building up a system.

Easy right....

Well actually the first few stages are quite easy, my first task will be to add some build commands for the kernel and get it building and running in QEMU. I will do the first stage with a initramfs only(I will cover what this is later) and I will add a microscopic statically linked init system into the mix as well to prevent the kernel rebooting straight away. At this point I will need to decide on a plan for what I want to achieve first and how...

[0] - https://code.google.com/p/git-repo/
[1] - http://git-scm.com/book/en/v2/Git-Tools-Submodules
[2] - https://github.com/rob-ward/intrepid-linux



2014-10-27

Okay folks, after several weeks(months(years???)) of thinking about it I have decided to create my own custom Linux[0] distro.

At this point I should probably clarify that this is not the first time that I have made my own custom distro. I work developing embedded Linux devices and as part of this I have developed custom Linux distros or built upon others custom distributions on several occasions. I have also personally built a number of simplified Linux distros in the past, usually for fun and to learn new technologies.

So what is different this time??? Well in the past the distros I have developed have all stopped at embedded device level, usable yes, but of limited functionality usually restricted to command line interfaces or running services i.e. web servers. In short, they have all had limited scope. This time I have decided that I want to create something on a larger scale, with nearly unlimited scope. I am not intending what I create to be of use for one specific purpose but rather as a personal playground where I can test and learn new technologies. In no way do I intend to create a Linux distro that you would download and install on your Grandparents PC. Hell, I don't really expect it to be something that anyone that isn't wanting to learn about or hack a Linux system will be even touching!

So what is the Plan? Well I suppose my plan is to come up with a plan. I do however have certain points already mapped out.

Firstly I am (best intentions) going to try and keep this blog up to date with the progress that I am making, along with anything useful I learn on the way. I am also planning to do as much in the open as possible.

The development will happen using git as my version control system, generally I use Gerrit[1] as my git backend, however at least for now I am indenting to use GitHub[2] to store and share my work in progress. The key plan is to make everything available.

The majority of the work, at least to start with is creating a make system that is able to create a bootable Linux+RootFS. To start with this will all be done using GNU Make[3], however I have several ideas that I would like to explore with regards to make systems so I will likely swap and change this at some point.

I will initially be using QEMU[4] for development but that can(and will) change. I am also aiming to make the system in such a way that it is able to cross compile. Cross compilation generally adds a lot of complexity, however it highlights issues with packages that you are building very early. It also means you end up fixing broken upstream make systems quite often. I must admit at this point that I am slightly fearful of attempting to get Xorg cross compiling, it was painful enough a few years ago when I compiled it natively... but it is just another challenge to overcome.

The system that I am building will need the ability to build itself so that will be one of the goals. I am also going to be using musl[5] as my c library, the primary reason for this is it looks like a nice project and I haven't used it for anything big as of yet. I will however aim to make the build system so it can support others(uClibc[6] and glibc[7] are the obvious ones but who knows).

The other thing I want to get right, and this relates back to potential alternatives to GNU make, is building in parallel as much as possible. When building something like this you should be able to use as many CPU cores as possible, I may even set it up for distcc[8] just for fun.

If I progress far enough I will have to start looking at storing parts of the builds as binaries, however that will be a long way down the road.

So that is it, the part of a plan that I have. As I figure more out I will update here, along with adding links to any repos with source in that I need.

[0] - https://www.kernel.org/
[1] - https://code.google.com/p/gerrit/
[2] - https://github.com/
[3] - http://www.gnu.org/software/make/
[4] - http://www.qemu.org
[5] - http://www.musl-libc.org/
[6] - http://www.uclibc.org/
[7] - http://www.gnu.org/software/libc/
[8] - https://code.google.com/p/distcc/