Re: Cirrus CI to shut down
Alan Somers <[email protected]>
| Newsgroups | gmane.os.freebsd.devel.hackers |
|---|---|
| Message-ID | <CAOtMX2j3rncaaVtp+owznDT8YHyrPy23sJ7dPJPkaOK3hGN8yA@mail.gmail.com> |
I tried the firecracker option. It does indeed boot faster! About one minute faster than the other qemu vm. But I found that it was too limited. The image is stripped down. It doesn't even include a compiler. I could install one from pkg, but that would probably wipe out the speed advantage. So I elected not to use it. Instead I went with vmactions/freebsd-vm. The maintainer is responsive and made i few improvements that I suggested. I've converted about two dozen repos over the last four days, and have a few left to go. But I'm still serious about the santa Claus thing. On Thu, Apr 16, 2026, 7:28 PM Alex Arslan <[email protected]> wrote: > This is quite sad news indeed. When I first came across Cirrus, I reached > out to Fedor Korotkov, their founder, to make a case for first-class > FreeBSD > support. I was surprised that he was actually interested, and ultimately > made it happen! I had worked a bit with Travis CI quite a while back as > they > brought up initial FreeBSD support, but it went live close to when they > moved to a credit-based system for builds and everybody migrated their > builds off of Travis. So Cirrus was really THE provider of hosted native > FreeBSD CI with a free tier. RIP. > > The Julia programming language project uses Buildkite for its CI for all > supported platforms, which includes FreeBSD x86-64. Buildkite overall works > quite well, and they even provide official Buildkite Agent release binaries > for FreeBSD x86-64. That said, the setup for it is complex and I admittedly > don't actually understand how Julia's is set up despite having contributed > the initial FreeBSD support, which I think speaks to Alan's point about it > being intimidating. But once it's set up, it's great. > > The Julia version multiplexer uses nested virtualization on GitHub Actions > for its CI, and we've had a number of issues with that. The most recent > issue I can recall was that the local disk space would be exhausted due to > some kind of QEMU garbage (I think it was), and forcing a cleanup to work > around that took about 5 extra minutes of CI time on every build. For > comparison, the builds on platforms that run natively on GHA take fewer > than 5 minutes start to finish. So it can be a bit painful. > > I haven't played with it much but someone wrote a GitHub Action for running > FreeBSD in a Firecracker VM[1], which touts a significantly improved boot > time in comparison to e.g. QEMU on GHA. > > [1]: https://github.com/acj/freebsd-firecracker-action > > > On Apr 13, 2026, at 9:08 AM, Alan Somers <[email protected]> wrote: > > > > On Sun, Apr 12, 2026 at 10:52 AM Moin Rahman <[email protected]> wrote: > >> > >> > >> > >>> On Apr 12, 2026, at 18:41, Alan Somers <[email protected]> wrote: > >>> > >>> On Sun, Apr 12, 2026 at 6:17 AM David Chisnall <[email protected]> > wrote: > >>>> > >>>> On 12 Apr 2026, at 12:56, Joe Schaefer <[email protected]> wrote: > >>>>> > >>>>> Good points. But I don’t mean host your own build farm. I mean use > the laptop your company gave you. > >>>> > >>>> There are many reasons not to do this. The top ones: > >>>> > >>>> - Our builds are done in a pristine environment. An attacker who > compromises a developer’s laptop can potentially commit malicious code to > the repo, but disabling force pushes means that we have an auditable view > of this. With builds done on the developers’ machine, they could just > inject malicious code into the binaries. > >>>> - PRs from external contributors may be malicious. Running them on a > cloud VM means that they have zero access to anything we care about and the > VM is destroyed at the end. Running them on a developer machine means > you’re just one container escape away from a flow from a GitHub PR to a > compromised developer laptop. > >>>> - Developer laptops are slower than big cloud machines. > >>>> - Developer laptops are a single architecture, doing cross builds is > annoying. > >>>> - Developer laptops have smaller disks, but ccache works *really* > nicely if it has a lot of space to store caches for every branch. > >>>> - Developers often submit PRs at the end of the day and close their > laptops, the cloud VM can start up when this happens. > >>>> > >>>> And you are talking about hosting your own build farm. That’s the > only other option. You’re just talking about running your build farm on > machines that are not always on and hold sensitive data. > >>>> > >>>> David > >>> > >>> I'm considering three options to replace Cirrus: > >>> > >>> 1) https://github.com/cross-platform-actions/action . This lets you > >>> run FreeBSD (among other platforms) on github workflows using nested > >>> virtualization. That's not ideal, but at least it looks easy. And it > >>> doesn't solve the cost problem that David brought up. > >>> > >>> 2) https://sourcehut.org/ . It supports CI on FreeBSD/amd64 . The > >>> code is open source, so a dedicated user could probably add whatever > >>> features he needs. Or, self-host. However, it's still officially in > >>> alpha, and I don't know how well it works. And the pricing strategy > >>> suggests that they're focused on amateurs. Sourcehut is intended as a > >>> full code-hosting service, but it can also be used just for CI, for > >>> projects hosted on Github, with > >>> https://github.com/emersion/hottub?tab=readme-ov-file . > >>> > >>> 3) Cirrus-Ci itself. Cirrus labs claimed in their blog post that > >>> they're going to open-source all of their code. So anybody else could > >>> resurrect the project. Since Cirrus doesn't own any of its own > >>> infrastructure, instead reusing AWS or GCE, it should be easy to > >>> resurrect it. Still, that's too much work for me alone. But I might > >>> be willing to help if somebody else takes the lead. > >>> > >>> 4) A fully personalized, serverless, cloud-hosted CI. It seems like > >>> it should be possible to build a personalized CI service that uses AWS > >>> lambda functions for coordination and UI, and uses AWS or GCE virtual > >>> machines for the CI workers. Very scalable. It would cost money to > >>> use, but not very much. This doesn't count towards my list of three > >>> because it doesn't exist yet. But it seems like an obvious idea. > >>> I'll ask Santa Claus for this next Christmas. > >>> > >> > >> I think Option 3 is not correct. They are going opensource only for > >> some of their products which includes Tart, Vetu and Orchard. > >> > >> I am working with one of my ex-client about something that I built > >> for them ages ago if we can goopensource. I will keep this list posted. > >> > >> Kind regards, > >> Moin > > > > I just learned about an additional option. BuildKite has native > > support for FreeBSD. There's a devel/buildkite-agent port that can be > > used for self-hosted CI workers, paired with BuildKite's cloud-based > > orchestration. There's also the Elastic CI stack, which spins up > > cloud-based workers on demand. The docs only mention Linux and > > Windows, but a BuildKite support engineer assured me that it will work > > for FreeBSD too. This option would no doubt have better performance > > than the nested-virtualization stuff. And probably better pricing > > too. But damn if it isn't intimidating to get started. Maybe > > CheriBSD could use it, but I fear that it's beyond the reach of a > > hobbyist. > > > > https://buildkite.com/docs/agent/self-hosted/aws/elastic-ci-stack > > https://buildkite.com/docs/agent/self-hosted/gcp/elastic-ci-stack > > > >