Project News

Project Atomic has been getting a great deal of attention since it was announced at Red Hat Summit this April. We saw a lot of enthusiasm at Summit for the concept of Atomic Hosts that are focused on running Docker containers, but still built from a well-known, well-tested distribution. 

One thing we’ve seen, though, is a little confusion about the separation between Project Atomic and Atomic Hosts. Project Atomic, itself, does not produce Linux OS releases. Instead, those will come from CentOS, Fedora, Red Hat, and potentially others.

The Value of Upstream First

Red Hat’s philosophy on driving technology innovation in upstream communities that then gets into the hand of customers in the form of robust products can be summarized as upstream first. 

Over the last decade this philosophy has spurred rapid technology innovation, fueled by user needs, that is ultimately delivered in supportable enterprise-grade products. This ensures that the best-suited technical solution is generated in the open source development community. 

Project Atomic is designed around this philosophy. Not only does it strive to be a community for the technology behind optimized container hosts, it is also designed to feed requirements back into the respective upstream communities. By leaving the downstream release of Atomic Hosts to the Fedora community, CentOS community and Red Hat, it can focus on driving technology innovation.

The Atomic Family

This is important to understand, because each of the offerings will differ in important ways, though they’ll also be very similar.

First, the release cycle will vary depending on where you’re getting your host. Release cycles are defined as a balance between the desire to deliver new features and enhancements and the ability of the target users to consume new releases. The release cycle for Red Hat Enterprise Linux Atomic Host (RHEL Atomic Host) will not match up with the release cycle for the Atomic offering from the Fedora Project. The RHEL Atomic Host release cycle will be suited for enterprise deployments, while the Fedora cycle will be more attuned to those doing leading edge development and testing newer technologies.

It also goes without saying that the actual content of the releases will also differ. Fedora’s offering will likely move faster, include newer and consequently less proven and robust technologies. These technologies may then work their way into future productized versions of RHEL Atomic Host, in the same way that Fedora feeds into future versions of Red Hat Enterprise Linux. 

How Fedora and CentOS Atomic offerings evolve is also up to the respective communities. The Fedora Cloud Special Interest Group (SIG) is still discussing exactly what packages will be included, how frequently releases will be generated, what type of images should be produced, and more. There’s plenty of room for the larger community to be involved in those projects, and we welcome participation. (See the Fedora wiki for more info, or join us on IRC (#fedora-cloud) on Freenode, or join the mailing list.)

Similar community discussion is happening in the CentOS community to land the optimal balance for their users.

The Role of Project Atomic

Project Atomic gives the larger community a place and tools to collaborate on related technologies that are vital to Atomic Hosts, such as SELinux (http://www.projectatomic.io/docs/docker-and-selinux/) to secure your containers, GearD (http://www.projectatomic.io/docs/geard/) for container wiring and systemd integration and Cockpit (http://www.projectatomic.io/docs/cockpit/) to help manage containers. It also gives users of the various releases a forum to solve problems on using the technologies, keep tabs on important developments, and to participate in developing the technologies. 

The project will also serve to disseminate news about Atomic Host distributions and important changes in the upstreams like Docker, rpm-ostree, SELinux, and others. So be sure to keep an eye on this site and @projectatomic.

As I’ve mentioned a number of times when I’ve spoken about Software Collections (SCLs), containers and SCLs are not mutually exclusive. In fact, SCLs promise to be really important for a lot of developers in building containers for production.

Langdon White has written up a great post on how to move an RHSCL app to Docker using an Atomic host:

Ok, now that Atomic is installed, I need to go make a container. After a few fits and starts, mostly related to pathing issues while using the RUN command, I got a decent Dockerfile working. You run this file by using the ‘docker build’ command. Later, you make the app go with the 'docker run’ command.

Check out the rest of Langdon’s post on the Red Hat Developer Blog.

Ever wondered if you can get systemd running in a Docker container? Apparently Dan Walsh did, and spent some time getting it to work.

While working with Docker, I looked at the great work that Scott Collier was doing for getting services to run within a container. Scott provides the fedora-dockerfiles package in docker with lots of “Dockerfile” examples. You can build Docker images by running “docker build” on these examples.

It seemed a little difficult, and wondered if getting systemd to run within a docker container, as I did with virt-sandbox-service, might make this simpler.

The Docker Model suggests that it is better to run a single service within a container. If you wanted to build an application that required an Apache service and a MariaDB database, you should generate two different containers. Some people insist on running multiple services within a container, and for this Docker suggested using the supervisord tool. In RHEL we do not want to support supervisord, since it is written in Python, and do not want to pull a Python requirement into containers, and it is just a package used to monitor multiple services. We already have a tool for monitoring multiple services called systemd.

After a little trial and error, Dan got systemd working within a container on Fedora Rawhide, and expects it will work on Fedora 20 or RHEL 7 (when it’s released). Give it a spin and let Dan know how it works out for you.

The response to the Project Atomic launch has been overwhelming, and we’re getting a lot of interest – and questions – about the project. In particular, many developers and admins want to know what sets Atomic apart from other Docker-focused offerings. 

A Distribution You Know and Trust

One of the most compelling reasons for users of Fedora, CentOS, and Red Hat Enterprise Linux is that the Atomic host brings the advantages of its base distribution. 

The packages are used to create an Atomic host bring the same advantages of the base distribution – you get the base features you’d get with Fedora, CentOS, or RHEL (like SELinux, filesystem options, kernel tuning, and so forth). For RHEL customers, there’s also the certification and training ecosystem you can rely on with Red Hat Enterprise Linux Atomic Host. 

Tools like Anaconda, which many admins and developers depend on for deploying and managing systems, will be available with Atomic hosts. This means Kickstart support, for instance, carries forward with Atomic hosts.

RPM Ecosystem

Because Atomic hosts are built with rpm-ostree, they benefit from the RPM ecosystem and will let projects and vendors distribute updates concurrently with the mainline distribution. So, for instance, when Fedora sends out an OpenSSL update the same RPMs used to deliver the update to users of the traditional Fedora distribution can be quickly used to generate a new tree for Atomic users. 

What’s more, users will have the ability to roll back updates when necessary.

Want to roll your own custom Atomic hosts? Want to do something crazy like including GNU Emacs on the Atomic Host? You can use the rpm-ostree technology to do just that.

GearD Support

A major feature that will be integrated into Atomic hosts is GearD, technology we’ve pulled in from the OpenShift Origin project, which helps bind systemd and Docker to make containers easier to configure and manage. 

Cockpit Support

Finally, Atomic hosts feature Cockpit, a server manager that makes managing hosts, services, and containers easier. Cockpit is also coming to Fedora, CentOS, and RHEL, giving you another standard tool to work with for all of your systems.

We’re Open

We are excited to show this work to the larger community, and to invite you to not only use Atomic – but we’re also excited about working with the larger community to make Atomic hosts the best way to run Docker containers. Join our developer community, and let’s make Atomic even better.

The application as shipping container metaphor behind the Docker project’s name and logo paints an attractive picture for developers: spawn a container on your local machine, fill it with code, and then ship it off to your far-flung users.

While the app is where the action happens, I can’t help but wonder what sort of ships await our containers when they arrive at the dock. No matter how well you’ve crafted or carefully you’ve packed your cargo, your application will only make it as far as its ship can take it.

Maybe the best thing about Project Atomic is the way it addresses both the shipping container and the ship itself, the former with Docker, and the latter with an intriguing (if less catchily-named) project called rpm-ostree.

We can use rpm-ostree to take a set of packages from one of our friendly neighborhood RPM-based Linux distributions, assemble it into a just-enough OS image, and then deploy and manage hosts based on that image in a manner that more closely matches the container model than do traditional OSes.

Project Atomic has kicked off with a host image, based on Fedora 20, that you can download and use right away, but if you’re like me, you’re looking for a way to build your own customized Atomic images. Here’s how to do it:

The Build Machine

For a build machine, I’ve been using a Fedora 20 minimal install with all updates. There are various configuration tweaks required on your build host for now, so it’s probably that you set aside a separate machine for this – I run it on a virtual machine.

Software Repositories

We have to install rpm-ostree, and some complementary packages, on our build host, as well as make available to our host the packages we intend to include in our Atomic image.

I’ve assembled the required packages for F20 in this Copr repository. (If you’re not familiar with Copr, you can learn about it here, but, more or less, it’s like Personal Package Archives for Fedora/CentOS.

cat > /etc/yum.repos.d/jasonbrooks-rpm-ostree.repo << EOF
[rpm-ostree]
name=Copr repo for rpm-ostree owned by walters
baseurl=http://copr-be.cloud.fedoraproject.org/results/jasonbrooks/rpm-ostree/fedora-20-x86_64/ 
gpgcheck=0
enabled=0
metadata_expire=1h
EOF

While an F20 build host already knows about F20 packages, rpm-ostree gets hung up on the $releasever variable that appears in the Fedora repo files by default. I have a separate repo file that pulls together the fedora, fedora-updates and fedora-updates-testing repos into a single repo file, with the release version hard-coded:

cat > /etc/yum.repos.d/fedora-20-atomic.repo << EOF
[fedora-20]
name=Fedora 20 x86_64
metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-20&arch=x86_64
enabled=0
gpgcheck=0
metadata_expire=1h

[fedora-20-updates]
name=Fedora 20 updates x86_64
metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f20&arch=x86_64
enabled=0
gpgcheck=0
metadata_expire=1h

[fedora-20-updates-testing]
name=Fedora 20 updates testing x86_64
metalink=https://mirrors.fedoraproject.org/metalink?repo=updates-testing-f20&arch=x86_64
enabled=0
gpgcheck=0
metadata_expire=1h
EOF

Build Machine Installation, Configuration

Next, let’s install some needed packages:

yum -y install --enablerepo rpm-ostree \
screen httpd yum-utils \
libgsystem \
ostree rpm-ostree-autobuilder \
binutils nss-altfiles

Then, we need to edit /etc/nsswitch.conf to include altfiles, like this:

passwd: files altfiles 
group:  files altfiles

Also, it’s necessary (for now) to disable SELinux on the build machine:

sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config && cat /etc/selinux/config

Now, we’ll set up a repository from which our eventual Atomic hosts will fetch upgrades:

mkdir /srv/rpm-ostree &&
cd /srv/rpm-ostree &&
mkdir -p repo &&
ostree --repo=repo init --mode=archive-z2 &&
cat > /etc/httpd/conf.d/rpm-ostree.conf <<EOF
DocumentRoot /srv/rpm-ostree
<Directory "/srv/rpm-ostree">
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>
EOF

Then, we’ll add a systemd service file for the rpm-ostree-autobuilder service, enable that service, along with httpd, and finally ensure that our firewall is open to http traffic:

cat > /etc/systemd/system/rpm-ostree-autobuilder.service <<EOF
[Unit]
Description=RPM-OSTree autobuilder

[Service]
WorkingDirectory=/srv/rpm-ostree
ExecStart=/usr/bin/rpm-ostree-autobuilder

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload &&
systemctl enable httpd &&
systemctl start httpd &&
systemctl reload httpd &&
systemctl enable rpm-ostree-autobuilder &&
systemctl start rpm-ostree-autobuilder &&
firewall-cmd --add-service=http &&
firewall-cmd --add-service=http --permanent

Now, reboot your build machine to let our SELinux-disabling take effect:

systemctl reboot

Tree-Building Time

Upon reboot, the rpm-ostree-autobuilder service should start, and begin looking, in the /srv/rpm-ostree directory, for a products.json file to define the trees and images to build.

A products.json that corresponds to the initial Project Atomic images is available in this gist. A products.json that produces multiple images (based on rawhide) is available in Colin Walters’ Fedora Atomic Initiative repository.

Some notes on products.json:

• The "repo" line of your products.json file should point to the repo directory on your build machine. If you want to use a separate machine to host updates, you can rsync the /srv/rpm-ostree/repo directory on your build machine to a separate server, and set that "repo" value accordingly.

• The "releases" line should include a list of yum repositories that match the names of the repos we configured above (or whatever other repos you want to use).

• The "products" section includes the "packages" specific to a particular image, and a "units" section listing systemd services to start by default.

The rpm-ostree-autobuilder will periodically check for products.json changes, and build accordingly, but you can spur a new build by running the command

rm -rf cache/raw/images images/auto cache/packageset*.txt && echo treecompose | rpm-ostree-autobuilder console

from your /srv/rpm-ostree directory. (that first, rm command is due to a caching bug and ought to be unnecessary before too long)

The ostree assembly process takes a while to complete. You can monitor the treecompose with:

tail -F /srv/rpm-ostree/tasks/treecompose/treecompose/log*

Eventually, the log will read Complete, and you can monitor the image-building with:

tail -F /srv/rpm-ostree/tasks/ensure-disk-caches/ensure-disk-caches/output.txt /srv/rpm-ostree/tasks/zdisks/zdisks/output.txt

When that completes, the log will say, Completed compression, renaming to… and you’ll be able to download your image in the tree (where exactly depends on your products.json) under YOURBUILDERHOST/images/auto/…/latest-qcow2.xz.

The builder produces qcow2 images. You can use the qemu-img tool to convert this image to raw, VDI (VirtualBox), VMDK (VMWare) and VHD (Hyper-V). Here’s how you convert from qcow2 to vdi:

xz -d latest-qcow2.xz && 
qemu-img convert -f qcow2 latest-qcow2 -O vdi latest.vdi

Upgrading, Atomically

Once you’re up and running in your new image, you can fetch updates with the command rpm-ostree upgrade. If you’ve followed along with me, step by step, you won’t have configured gpg signing, and rpm-ostree will tell you to add gpg-verify=false to your remote configuration. The relevant config file lives at /ostree/repo/config.

Of course, you won’t have any upgrades available right out of the gate, but once enough time goes by for new packages to enter Fedora’s update repo, or, once you’ve added packages to your products.json and run the treecompose command above, you’ll be able to pull down these changes with rpm-ostree upgrade, followed by a reboot.

Don’t like the changes? You can rollback with rpm-ostree rollback, followed by a reboot.

Till Next Time

If you run into trouble following this walkthrough, I’ll be happy to help you get up and running or get pointed in the right direction. Ping me at jbrooks in #atomic on freenode irc or @jasonbrooks on Twitter. Also, be sure to check out the Project Atomic Q&A site.