July 22, 2017

0.4.5Kristaps Dzonsons

Allow for data-sblg-set-xxxx="yyyy" attributes for custom key-value pairs. These may later be extracted using the ${sblg-get|xxxx} invocation. This is very useful for having structured values within, say, navigation.

Also merge Reyk Flöter's data-sblg-navxml tag to cause the contents of a data-sblg-nav="1" statement to be included verbatim, not within a list.

July 21, 2017

books chapter fiveTed Unangst (tedu@)

A few different perspectives this week.

coders

All the previous interviewers were generally kind of down on Java, but now we’re talking to Joshua Bloch, Java Architect at Google, who’s pretty obviously down with Java. He also likes the Design Patterns book, in stark contrast to some previous subjects.

As far as programming goes, in any language, he spends a lot of time just getting the names of identifiers right. Always keep a dictionary handy. Everybody knows one letter names are bad, but this is the first I’ve read about this level of attention. That kind of sounds excessive at first, but after some reflection I think he’s on to something. I can recall a few times when a function was roughly named correctly, but there was enough wiggle room for misunderstanding. Java is infamous for preposterously long names, but I suspect it’s possible to conjure up some precise short names as well.

We spend some time talking about the coming age of concurrency and how Java is best suited to tackle it, and the rising use of Java within Google to replace some C++ code, and the burden of adding generics to Java. It reads not just a little, but a lot, like an interview today with someone from the Go team. His advocacy for Java concurrency seems pretty weak. They’ve got lowlevel primitives, but also ConcurrentHashMap, which to be honest doesn’t seem all that high level. You look around at Go or Rust today, and they’re taking a different approach. His comments on generics read exactly like what one would read today about Go. You didn’t really need them to write code, then they added them and the language became far more complicated than anybody expected. The more things change.

One of the hardest problems he worked on turned out to be what we now call stack clash. A debug library would occasionally print the wrong thread ID in its messages. This was ignored as inconsequential. Finally, other problems arose, and after some debugging they found that an enormous stack variable was causing the stack pointer to jump way past the red zone and into another thread’s stack. A valuable lesson in not ignoring inconsequential bugs without understanding the cause.

A colleague apparently mentioned that problems like this are why it’s important to understand how things work all the way down. He counters that it’s a reason to use safe languages (I believe the bug happened in C code), but that sounds like exactly the reason people need to study what’s happening beneath the abstraction. A “safe” language that doesn’t do stack probing will have the exact same issue.

founders

Tim Brady was the first employee at Yahoo. Yahoo got its start as a simple catalog of links that Jerry Yang and David Filo were collecting, mostly for their EE research. Then people sent them links, and the catalog grew and grew. I liked this as a slight variation of the typical advice to build a product that solves a problem you have. They built something, a very little something, that solved their problem but then they let other people use it.

For a long time, Yahoo ignored full text search. They’d search their directory, and if that failed, use this partner or that partner to search the rest of the web. This worked well enough when full text search was kinda bad, and they could swap out one partner for another. Then Google came along and made search work, far better than before, and Yahoo was done. There’s a tipping point where the lesser product gets so much better that it becomes the better product.

Mike Lazaridis founded Research In Motion, foresaw the rising important of wireless data and mobile email, and made the BlackBerry, which dominates the market. Maybe revise that last part for the second printing.

Very early on the company’s history, they had an opportunity to persue a contract for the Canadian space agency, which had always been a dream of Tim’s. After the prototypes would be approved, there would be a mass production order for... six. He turned that down to focus on wireless, which he was sure would eventually become much more popular. And indeed, it has. BlackBerry was even used by NASA, so in the long run he fulfilled his space dream anyway.

He attributes a great deal of BlackBerry’s success to consistency, avoiding the fads, keeping the product simple. And I think that’s true up to a point. You learned how to use a BlackBerry and you were set. But then other fruits came along, and kind of like Yahoo, they found themselves obsolete before they had much of a chance to pivot.

man-month

Everybody has heard the tale of the second system. All the stuff which was reasonably cut from the first system gets unreasonably piled into the second system. The very worst second systems are those designed by a whole team of architects who have each designed one previous system, which seems to the the natural tendency. You’ve built X, Y, and Z, and now you want to build the successor system, XYZ, which combines and unites these disparate systems. But everybody has their favorite features which got cut, and are essential to add to the next version. Brooks proposes self discipline as the cure, but I also wonder if this isn’t an argument that we should let separate products evolve a little longer on their own. Build X2, Y2, Z2, and so forth, and then see what features truly are essential. Only combine finished products, not works in progress.

Which gets into the second half of the second system effect, when obsolete functionality is refined beyond its economic benefit. He gives as an example the linkage editor, which I believe we’d call ld, and its support for overlays. Apparently the OS/360 overlay system was the greatest ever built, but the technique was somewhat obsolete by the time it shipped, in favor of dynamic linking. Alas, the overlay support itself, fine as it is, dramatically slows down the linker. Recognize when to let go and focus effort on building other features.

On specifications, several important traits are identified. It must be precise. It must be consistent. It must be complete. One strategy is to have only one or two people write the final specification. They can work from the notes of many others, but the final product should be in their own voice.

Many specifications have a formal and prose version. It’s essential that one specify which has precedence. Beware the dangers of specification by implementation. It’s often easier to change the manual than the code, but that doesn’t result in the best product. A good solution is to build multiple implementations in parallel. This makes it easier to detect discrepancies and also encourages making the right fix. He includes a few examples of systems where unspecified behavior eventually leaked into a de facto spec because there was only one implementation, such as the contents of a register after a particular operation. I’ve seen this plague just about every language, where building a second implementation eventually resorts to running tests against the first implementation instead of reading the manual.

pragmatic

A chapter on tools. Craftsmen love their tools and always use the best.

14. The power of plain text is that it’s easy to work with. Source code, obviously, but also configuration files, data files, etc. Its lowest common denominator nature means you’ll always have the tools to work with it, even when things have gone terribly wrong.

15. Learn to use the shell, instead of clicking in the GUI. I don’t disagree, but there’s some cherry picking of examples here.

16. Learn to use a powerful text editor and use it for everything. Code, email, administration, etc. I happen to use vim for a great many things, but not everything. And I think this advice might get pretty awkward for someone using gmail and an IDE. Not actually advice I would follow unquestioningly. Veering a little bit into myth here.

17. Use source control. Can’t disagree with this, but the few remaining cavemen who don’t yet use source control probably aren’t going to start anytime soon.

code

We’re going to learn about some bits, bit by bit. Paul Revere used two lanterns, or bits, to convey information about the British invasion. 00 for no action, 01 for land, 11 for sea. But what about 10? That’s also for land. It can be hard to see an unlit lantern at a distance at night, so we’ve added some redundancy.

We can also use bits to encode movie ratings, from A+ all the way down to Pauly Shore. Or film speed. Or UPCs. UPC is a good case study, because there’s a lot of redundancy. The first few bits always follow the same pattern, as do the last bits. This lets you tell if you’re reading the code forwards or backwards. And there’s some check bits as well for parity.

Stopping here, since the next two chapters go together.

coda

From Yahoo to RIM to OS/360, a lot of companies invested in obsolete technology. Classic disruption, where something that didn’t work gets better and eventually replaces the dominant technique. Do we include Java in the group as well?

DragonFly and node.jsDragonFly BSD Digest

If you’ve had odd behavior with node.js (which I have) on DragonFly, it may be fixed now.

Ted Unangst on notable recent changes in OpenBSDUndeadly

The flak reports by Ted Unangst (tedu@) continue with part 624.

Update - part 625

openbsd changes of note 625Ted Unangst (tedu@)

Halcyon changes of summer.

Continue with some cleanup and improvement of the depend step of building. Lots of little things to support lex and yacc better as well.

Intel Optane parts are leaking into the wild, some driver fixes to support them.

Add support for pattern substitution to variables in ksh using a common syntax borrowed from ksh93. Or not, reverted.

Deprecate fgetln.

Add detection for missing X sets to syspatch.

Refinement of the inteldrm code, including better backlight support.

A special edition of slaacd for the installer.

After much wailing and gnashing of teeth, fix strtol to parse strings like “0xridiculous”.

A fix for malloc and zero sized allocations when using canaries.

Add the ability to pause and unpause VMs in vmd.

Remove “listen secure” syntax from smtpd.conf. It’s broken since a couple of months and noone complained.

Remove sending of router solicitations and processing of router advertisements from the kernel.

The lidsuspend sysctl has been fully replaced by lidaction.

Fix fortune to filter out unprintable characters. Convert the fortune files to using UTF-8 instead of archaic overprinting. Fortunes with unprintable words may still be obtained with the -o option.

Introduce some quirks to the IDE and ATA code to prevent drives from attaching twice on hyper-v.

Add vmctl send and receive as well.

Update to xterm 330.

Remove some magic cleanup from dhclient. It will not deliberately attempt to interfere with other operations on the same interface.

Update libexpat to 2.2.2. Fixes NULL parser dereference.

Ilja Van Sprundel found a whole mess of kernel bugs in this and that. Some info leaks, some erroneous signal handling, some unbounded malloc calls. Lions, tigers, bears. Try to fix them.

New package, p5-Test-Deep-Type-0.008OpenBSD packages
a Test::Deep plugin for validating type constraints
New package, p5-Check-ISA-0.09OpenBSD packages
correct check for object classes in Perl
BSDNow 203: For the love of ZFSDragonFly BSD Digest

You can guess what BSDNow is about this week, can’t you?   Well, there’s more than just ZFS, though there’s an excellent historical summary on the site.

New package, qca-qt5-2.1.3OpenBSD packages
Qt Cryptographic Architecture

July 20, 2017

For the love of ZFS | BSD Now 203BSD Now
This week we clear up some ZFS FUD, show you how to write a NetBSD kernel module, cover DragonflyBSD on the desktop & more!
0.4.4Kristaps Dzonsons

Fix <id> element in Atom feeds as patched by Reyk Flöter. Thanks!

July 19, 2017

ACPICA 20170629 update in DragonFlyDragonFly BSD Digest

Sascha Wildner has updated ACPICA in DragonFly to Intel’s version 20170629.  This will be of most interest to those with newer motherboards, as it matches ACPI 6.2.

Blog about my blogReyk Floeter (reyk@)

I really like Twitter because it allows me to share short messages, we have a great community, and 140 characters are enough for everybody.

And this statement was exactly 140 characters, but sometimes I want to say more than that. And that's why I finally created this new blog. I was never really into blogging because I barely had time or the audience to write long articles. I sometimes wrote short stories for sites like undeadly.org, I collected some of them here, but my own blog was hosted on tumblr and never saw any activity.

I want to try it again, and this time I decided to create a self-hosted blog. Something that runs on my own server and with httpd, the web server that I wrote for OpenBSD. So I was looking for potential blogging tools that I could use to run my own blog. Besides the popular and heavyweight ones such as WordPress, there are countless other options: I looked at blogs from fellow developers, such as Ted Unangst's flak (I like the fact that it is written in Lua but the implementation is a bit over my head), or Pelican that is used by Peter Hessler for bad.network (but, sorry, I don't like Python), and finally Kristaps Dzonsons' sblg that is used for all of his projects and blogs. I decided to use sblg.

Blog about my blogReyk Floeter (reyk@)

I really like Twitter because it allows me to share short messages, we have a great community, and 140 characters are enough for everybody.

And this statement was exactly 140 characters, but sometimes I want to say more than that. And that's why I finally created this new blog. I was never really into blogging because I barely had time or the audience to write long articles. I sometimes wrote short stories for sites like undeadly.org, I collected some of them here, but my own blog was hosted on tumblr and never saw any activity.

I want to try it again, and this time I decided to create a self-hosted blog. Something that runs on my own server and with httpd, the web server that I wrote for OpenBSD. So I was looking for potential blogging tools that I could use to run my own blog. Besides the popular and heavyweight ones such as WordPress, there are countless other options: I looked at blogs from fellow developers, such as Ted Unangst's flak (I like the fact that it is written in Lua but the implementation is a bit over my head), or Pelican that is used by Peter Hessler for bad.network (but, sorry, I don't like Python), and finally Kristaps Dzonsons' sblg that is used for all of his projects and blogs. I decided to use sblg.

July 18, 2017

Dports, Rust, and updatesDragonFly BSD Digest

I’ve waited to post this because it’s a bit complicated, but here is the summary: dports didn’t get updated with new binary builds for a while because Rust stopped working, which killed Firefox.  Michael Neumann got Rust working again, and packages are updated.

(Use -f if you have upgrade troubles.)

moving to httpsTed Unangst (tedu@)

The time has finally come to switch everything to https. Actually, I’ve been using https for a while, but now it’s time to inflict, er invite, everyone else along for the ride.

There is some security benefit, of course, but really it’s all about the speed. I want flak to be as fast as possible, thus we need to be using the fastest protocol.

On the security front, however, there may be a few things to mention. Curiously, some browsers react to the addition of encryption to a website by issuing a security warning. Yesterday, reading this page in plaintext was perfectly fine, but today, add some AES to the mix, and it’s a terrible menace, unfit for even casual viewing. But fill out the right forms and ask the right people and we can fix that, right?

I find it strange, to say the least, that I am required to request permission from the authorities to make a secure web site. When I first created flak, there was very little paperwork. I started a server, and that was that. Do we really want an internet where the use of encryption requires authorization?

What if the authorities say no? Maybe not because they find me personally objectionable, though that’s certainly worth considering, but what happens when they inevitably fuck up some simple thing like leap seconds or URL parsing or whatever? In my experience, reliability is not increased by increasing external dependencies.

Having opted out of having the authorities say I’m me, can I opt out of having them say anybody else is me? Alas, no. There is a secret browser handshake to partially opt out, but the wrinkle is that it first requires opting in. No way to actually decline the whole mess.

But not all is lost. I have created my very own certificate of great authenticity. (sha256 on the front page.) Before rushing to install, however, one might consider the consequences. What if I go rogue and sign a bunch of other sites I’m not supposed to? A reasonable concern if you don’t know me. But how well do you know the 300 people controlling the certs you do trust? Do you even know their names?

Or maybe the creation of the rogue cert is not deliberate, but accidental. My signing protocol is currently “try not to fuck up too bad” which is probably shorter than the elaborately documented procedures and safeguards some other people use. Thus, we have the setup for a grand experiment. Who will improperly sign a certificate for a domain they don’t own or release a “testing only” cert into the wild first, me or someone else?

To swing, briefly, back to the good news, my cert uses the name constraint extension, so in theory it should only be valid for my domain and of little risk to the internet at large. Like much of X.509, this seems slightly backwards: as the user about to install this cert, you should be telling the cert for what sites you trust it, instead of the cert telling you what’s trustworthy. Be aware that software support for name constraints is somewhat hit or miss, and it fails open, so make sure to consult the user manual for your browser. I’m sure it mentions name constraints in there somewhere. (I could, should?, mark the constraints as critical which in theory would mean that unsupported software would fail closed, which would indeed be safer, but then it becomes very difficult to override that, returning to the idea that too much information about the trustiness of a cert is contained within the cert itself. The absence of the critical marker does’t affect or degrade security for modern software.)

So how does one verify that the downloaded cert is the original? The same way the CAs do. Perform a DNS lookup, make a web request, trust the result. The addition of HPKP would indicate that people find the CA model untrustworthy, solving the problem with trust on first use key continuity. Why not cut out the middle man? Protesting the CAs is admittedly pretty futile, but if I can’t do it, who can?

One difficult wrinkle is that not everyone controls their own trust chain. A remote service RSS reader may not even offer the option to modify its https behavior. The perils of depending on software one doesn’t control. For now, RSS URLs are exempted from the redirect, but eventually that will be sealed off as well.

Automated sentiment analysis: good news with a side of bad news with a side of good news with a side of bad news with a side of good news with a side of bad news with a side of good news with a side of bad news. Most likely topical match: TLS.

OpenBSD and the modern laptopPeter Hansteen
Did you think that OpenBSD is suitable only for firewalls and high-security servers? Think again. Here are my steps to transform a modern mid to high range laptop into a useful Unix workstation with OpenBSD.

One thing that never ceases to amaze me is that whenever I'm out and about with my primary laptop at conferences and elsewhere geeks gather, a significant subset of the people I meet have a hard time believing that my laptop runs OpenBSD, and that it's the only system installed.

A typical exchange runs something like,
"So what system do you run on that laptop there?"
"It's OpenBSD. xfce is the window manager, and on this primary workstation I tend to just upgrade from snapshot to snapshot."
"Really? But ..."
and then it takes a bit of demonstrating that yes, the graphics runs with the best available resolution the hardware can offer, the wireless network is functional, suspend and resume does work, and so forth. And of course, yes, I do use that system when writing books and articles too. Apparently heavy users of other free operating systems do not always run them on their primary workstations.

I'm not sure at what time I permanently converted my then-primary workstation to run OpenBSD exclusively, but I do remember that when I took delivery of the ThinkPad R60 (mentioned in this piece) in 2006, the only way forward was to install the most recent OpenBSD snapshot. By mid-2014 the ThinkPad SL500 started falling to pieces, and its replacement was a Multicom Ultrabook W840, manufactured by Clevo. The Clevo Ultrabook has weathered my daily abuse and being dragged to various corners of the world for conferences well, but on the trek to BSDCan 2017 cracks started appearing in the glass on the display and the situation worsened on the return trip.

So the time came to shop around for a replacement. After a bit of shopping around I came back to Multicom, a small computers and parts supplier outfit in rural Åmli in southern Norway, the same place I had sourced the previous one.

One of the things that attracted me to that particular shop and their own-branded offerings is that they will let you buy those computers with no operating system installed. That is of course what you want to do when you source your operating system separately, as we OpenBSD users tend to do.

The last time around I had gone for a "Thin and lightweight" 14 inch model (Thickness 20mm, weight 2.0kg) with 16GB RAM, 240GB SSD for system disk and 1TB HD for /home (since swapped out for a same-size SSD, as the dmesg will show).

Three years later, the rough equivalent with some added oomph for me to stay comfortable for some years to come ended me with a 13.3 inch model, 18mm and advertised as 1.3kg (but actually weighing in at 1.5kg, possibly due to extra components), 32GB RAM, 512GB SSD and 2TB harddisk. For now the specification can be viewed online here (the site language is Norwegian, but product names and units of measure are not in fact different).

That system arrived today, in a slender box:



Here are the two machines, the old (2014-vintage) and the new side by side:



The OpenBSD installer is a wonder of straightforward, no-nonsense simplicity that simply gets the job done. Even so, if you are not yet familiar with OpenBSD, it is worth spending some time reading the OpenBSD FAQ's installation guidelines and the INSTALL.platform file (in our case, INSTALL.amd64) to familiarize yourself with the procedure. If you're following this article to the letter and will be installing a snapshot, it is worth reading the notes on following -current too.

The main hurdle back when I was installing the 2014-vintage 14" model was getting the system to consider the SSD which showed up as sd1 the automatic choice for booting (I solved that by removing the MBR, setting the size of the MBR on the hard drive that showed up as sd0 to 0 and enlarging the OpenBSD part to fill the entire drive).

Let's see how the new one is configured, then. I try running with the default UEFI "Secure boot" option enabled, and it worked.

Here we see the last part of the messages that scroll across the screen when the new laptop boots from the USB thumbdrive that has had the most recent OpenBSD/amd64 install61.fs dd'ed onto it:



And as the kernel messages showed us during boot (yes, that scrolled off the top before I got around to taking the picture), the SSD came up as sd1 while the hard drive registered as sd0. Keep that in mind for later.



After the initial greeting from the installer, the first menu asks what we want to do. This is a new system, so only (A)utoinstall and (I)nstall would have any chance of working. I had not set up for automatic install this time around, so choosing (I)nstall was the obvious thing to do.

The next item the installer wants to know is which keyboard layout to use and to set as the default on the installed system. I'm used to using Norwegian keyboards, so no is the obvious choice for me here. If you want to see the list of available options, you press ? and then choose the one you find the must suitable.

Once you've chosen the keyboard layout, the installer prompts you for the system's host name. This is only the host part, the domain part comes later. I'm sure your site or organization has some kind of policy in place for choice of host names. Make sure you stay inside any local norms, the one illustrated here conforms with what we have here.

Next up the installer asks which network interfaces to configure. A modern laptop such as this one comes with at least two network interfaces: a wireless interface, in this case an Intel part that is supported in OpenBSD with the iwm(4) driver, and a wired gigabit ethernet interface which the installer kernel recognized as re0.

Quite a few pieces the hardware in a typical modern laptop requires the operating system to load firmware onto the device before it can start interacting meaningfully with the kernel. The Intel wireless network parts supported by the iwm(4) driver and the earlier iwn(4) all have that requirement. However, for some reason the OpenBSD project has not been granted permission to distribute the Intel firmware files, so with only the OpenBSD installer it is not possible to use iwm(4) devices during an initial install. So in this initial round I only configure the re0 interface. During the initial post-install boot the rc.firsttime script will run fw_update(1) command that will identify devices that require firmware files and download them from the most convenient OpenBSD firmware mirror site.

My network here has a DHCP server in place, so I simply choose the default dhcp for IPv4 address assignment and autoconf for IPv6.

With the IPv4 and IPv6 addresses set, the installer prompts for the domain name. Once again, the choice was not terribly hard in my case.



On OpenBSD, root is a real user, and you need to set that user's password even if you will rarely if ever log in directly as root. You will need to type that password twice, and as the install documentation states, the installer will only check that the passwords match. It's up to you to set a usefully strong password, and this too is one of the things organizations are likely to have specific guidelines for.

Once root's password is set, the installer asks whether you want to start sshd(8) by default. The default is the sensible yes, but if you answer no here, the installed system will not have any services listening on the system's reachable interfaces.

The next question is whether the machine will run the X Windows system. This is a laptop with a "Full HD" display and well supported hardware to drive it, so the obvious choice here is yes.

I've gotten used to running with xenodm(1) display manager and xfce as the windowing environment, so the question about xenodm is a clear yes too, in my case.

The next question is whether to create at least one regular user during the install. Creating a user for your systems adminstrator during install has one important advantage: the user you create at this point will be a member of the wheel group, which makes it slightly easier to move to other privilege levels via doas(1) or similar.

Here I create a user for myself, and it is added, behind the scenes, to the wheel group.

With a user in place, it is time to decide whether root will be able to log in via ssh. The sensible default is no, which means you too should just press enter here.

The installer guessed correctly for my time zone, so it's another Enter to move forward.

Next up is the part that people have traditionally found the most scary in OpenBSD installing: Disk setup.

If the machine had come with only one storage device, this would have been a no-brainer. But I have a fast SSD that I want to use as the system disk, and a slightly slower and roomier rotating rust device aka hard disk that I want primarily as the /home partition.

I noted during the bsd.rd boot that the SSD came up as sd1 and the hard drive came up as sd0, so we turn to the SSD (sd1) first.

Since the system successfully booted with the "Secure boot" options in place, I go for the Whole disk GPT option and move on to setting partition sizes.

The default suggestion for disk layout makes a lot of sense and will set sensible mount options, but I will be storing /home on a separate device, so I choose the (E)dit auto layout option and use the R for Resize option to redistribute the space left over to the other partitions.

Here is also where you decide the size of the swap space, traditionally on the boot device's b partition. Both crashdumps and suspend to disk use swap space for their storage needs, so if you care about any of these, you will need to allocate at least as much space as the amount of physical RAM installed in the system. Because I could, I allocated the double of that, or 64GB.

For sd0, I once again choose the Whole disk GPT option and make one honking big /home partition for myself.

The installer then goes on to create the file systems, and returns with the prompt to specify where to find install sets.

The USB drive that I dd'ed the install61.fs image to is the system's third sd device (sd2), so choosing disk and specifying sd2 with the subdirectory 6.1/amd64 makes sense here. On the other hand, if your network and the path to the nearest mirror is fast enough, you may actually save time choosing a http network install over installing from a relatively slow USB drive.

Anyway, the sets install proceeds and trundles through what is likely the longest period of forced inactivity that you will have during an OpenBSD install.

The installer verifies the signed sets and installs them.



Once the sets install is done, you get the offer of specifying more sets -- your site could have a site-specific items in an install set -- but I don't have any of those handy, so I just press enter to accept the default done.

If you get the option to correct system time here, accept it and have ntpd(8) set your system clock to a sane setting gleaned from well known NTP servers.

With everything else in place, the installer links the kernel with a unique layout, in what is right now a -current-only feature, but one that will most likely be one of the more talked-about items in the OpenBSD 6.2 release some time in the not too distant future.

With all items on the installer's agenda done, the installer exits and leaves you at a root shell prompt where the only useful action is to type reboot and press enter. Unless, of course you have specific items you know will need to be edited into the configuration before the reboot.

After completing the reboot, the system did unfortunately not, as expected, immediately present the xenodm login screen, but rather the text login prompt.

Looking at the /var/log/Xorg.0.log file pointed to driver problems, but after a little web searching on the obvious keywords, I found this gist note from notable OpenBSD developer Reyk Flöter that gave me the things to paste into my /etc/xorg.conf to yield a usable graphics display for now.

My task for this evening is to move my working environment to new hardware, so after install there are really only two items remaining, in no particular order:
  • move my (too large) accumulation of /home/ data to the new system, and
  • install the same selection of packages on the old machine to the new system.
The first item will take longer, so I shut down all the stuff I normally have running on the laptop such as web browsers, editors and various other client programs, and use pkg_info(1) to create the list of installed packages on the 'from' system:

$ pkg_info -mz >installed_packages

then I transfer the installed_packages file to the fresh system, but not before recording the df -h status of the pristine fresh install:

$ df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/sd1a     1005M   76.4M    878M     8%    /
/dev/sd0d      1.8T    552K    1.7T     0%    /home
/dev/sd1d     31.5G   12.0K   29.9G     0%    /tmp
/dev/sd1f     98.4G    629M   92.9G     1%    /usr
/dev/sd1g      9.8G    177M    9.2G     2%    /usr/X11R6
/dev/sd1h      108G    218K    103G     0%    /usr/local
/dev/sd1k      9.8G    2.0K    9.3G     0%    /usr/obj
/dev/sd1j     49.2G    2.0K   46.7G     0%    /usr/src
/dev/sd1e     98.4G    5.6M   93.5G     0%    /var

Not directly visible here is the amount of swap configured in the sd1b partition. As I mentioned earlier, crashdumps and suspend to disk both use swap space for their storage needs, so if you care about any of these, you will need to allocate at least as much space as the amount of physical RAM installed in the system. Because I could, I allocated the double of that, or 64GB.

I also take a peek at the old system's /etc/doas.conf and enter the same content on the new system to get the same path to higher privilege that I'm used to having. With those in hand, recreating the set of installed packages on the fresh system is then a matter of a single command:

$ doas pkg_add -l installed_packages

and pkg_add(1) proceeds to fetch and install the same packages I had on the old system.

Then there is the matter of transferring the I-refuse-to-admit-the-actual-number-of gigabytes that make up the content of my home directory. In many environments it would make sense to just restore from the most recent backup, but in my case where the source and destination sit side by side, i chose to go with a simple rsync transfer:

$ rsync  -rcpPCavu 192.168.103.69:/home/peter . | tee -a 20170710-transferlog.txt

(Yes, I'm aware that I could have done something similar with nc and tar, which are both in the base system. But rsync wins by being more easily resumable.)

While the data transfers, there is ample time to check for parts of the old system's configuration that should be transferred to the new one. Setting up the hostname.iwm0 file to hold the config for the wireless networks (see the hostname.if man page) by essentially copying across the previous one is an obvious thing, and this is the time when you discover tweaks you love that were not part of that package's default configuration.

Some time went by while the content transferred, and I can now announce that I'm typing away on the machine that is at the same time both the most lightweight and the most powerful machine I have ever owned.

I am slowly checking and finding that the stuff I care about just works, though I haven't bothered to check whether the webcam works yet. I know you've been dying to see the dmesg, which can be found here. I'm sure I'll get to the bottom of the 'not configured' items (yes, there are some) fairly soon. Look for updates that will be added to the end of this column.

And after all these years, I finally have a machine that matches my beard color:



If you have any questions on running OpenBSD as a primary working environment, I'm generally happy to answer but in almost all cases I would prefer that you use the mailing lists such as misc@openbsd.org or the OpenBSD Facebook group so the question and hopefully useful answers become available to the general public. Browsing the slides for my recent OpenBSD and you user group talk might be beneficial if you're not yet familiar with the system. And of course, comments on this article are welcome.


Update 2017-07-18: One useful thing to do once you have your system up and running is to submit your dmesg to the NYCBUG dmesg database. The one for the system described here is up as http://dmesgd.nycbug.org/index.cgi?do=view&id=3227
Removed, zh-crxvt-2.10.2Removed OpenBSD packages
Chinese(Big5) VT100 terminal emulator for X
New package, phonon-qt5-4.9.1OpenBSD packages
multimedia layer for Qt4/KDE4
New package, borgmatic-1.0.3OpenBSD packages
wrapper for Borg to create and prune backups

July 17, 2017

New package, py-fasteners-0.14.1OpenBSD packages
python package that provides useful locks

July 16, 2017

Add vmctl send and vmctl receiveUndeadly
As we see from the commit message, new developer Pratik Vyas (pd@) adds the ability to do paused VM migrations for VMM.

Mike Larkin also writes on Twitter:

#OpenBSD vmctl(8) : vmctl send myvm | ssh mlarkin@somewhere.com vmctl receive Yes, it's that easy.
Ed: this is *paused* migrations, not live migrations.