August 16, 2017

How to change or configure OpenBSD package install mirrornixCraft
openbsd commandI am trying to install nginx server on OpenBSD but keep getting an error that read as follows when I run the pkg_add command:
https://mirror.leaseweb.com/pub/OpenBSD/6.1/packages/amd64/quirks-2.304.tgz" title="quirks-2.304.tgz: ftp: Error retrieving file: 404 Not Found signify: gzheader truncated https://mirror.leaseweb.com/pub/OpenBSD/6.1/packages/amd64/nginx-1.10.2p2.tgz" title="nginx-1.10.2p2.tgz: ftp: Error retrieving file: 404 Not Found signify: gzheader truncated

How do I change or configure OpenBSD package install mirror for the pkg_add command?
Removed, p5-Test-Harness-3.36Removed OpenBSD packages
Run Perl standard test scripts with statistics
Removed, p5-I18N-LangTags-0.35Removed OpenBSD packages
perl support to RFC 3066 language tags

August 15, 2017

t2k17 Hackathon Report: Bob Beck on buffer cache tweaks, libressl and pledge progressUndeadly
The first report from the just completed t2k17 hackathon comes from Bob Beck, who writes:

Unusually I had basically nothing to do with organizing this year, I let krw@ do all the dirty work, which was good since life has been a bit crazy over the last couple months with things keeping me from openbsd.
Read more...
New package, speexdsp-1.2rc3OpenBSD packages
speech processing DSP library
Removed, p5-Test-Tester-0.109Removed OpenBSD packages
ease testing test modules built with Test::Builder
Removed, mico-2.3.13Removed OpenBSD packages
free and complete CORBA-compliant implementation

August 13, 2017

Lazy Reading for 2017/08/13DragonFly BSD Digest

Overflow!

books chapter eightTed Unangst (tedu@)

Smaller is better.

coders

Peter Norvig has been around for a long time, writing lots of code. Back in the day, a lot of programming was done solo, and you wrote everything from scratch. Now with something like Ruby on Rails, it’s very important to understand the interfaces rather than the insides.

Peter doesn’t really like pair programming much, although he has an interesting alternative. Two people discuss a problem for a while, then independently write code, then they swap positions and test each other’s code. This sounds like a good way to keep output high, but also benefit from lots of review and it ensures that there’s always two people who understand any piece of code. We talk a bit about the IBM master-programmer idea, Brooks’s surgical team, which Peter initially calls the dumbest idea ever. “Why would anybody want to subject themselves to being a gopher for the one real programmer?” Indeed, nobody wants to be (or admit to being) anything less than the best. But maybe it works if it’s part of an apprenticeship? In particular, debugging skills are rarely taught formally, but picked up along the way. You could learn much faster by watching an experienced debugger work than by stumbling along oneself.

What does it mean for a program to be correct, at least in the formal logic sense? Are the first ten results Google returns for a search provably the best ten results? “I think once you start solving these types of problems or the problem of having a robot car navigate through a city without hitting anybody, the logical proof gets thrown out pretty quickly.” So that actually sounds kind of ominous. Not to mention topical. I’d really prefer that your self driving car not run me over, but who knows what the neural nets are thinking?

Peter is another fan of small programs. If you can keep the whole idea in your head, that’s more likely to work than a larger program. He also recommends reading some algorithm books, so you know what choices are available for problem solving. He mentions a few times you need to know about backtracking and dynamic programming to solve certain problems, and without that knowledge you’re going to have a very hard time. I think this makes a lot of sense. Every problem has a certain shape, if we step back and look at it, and then the solution is likely to be already known. This requires both that we be able to see the entire problem and that we have knowledge of available solutions.

One time Peter was at Heathrow airport and the power was out. But nevertheless things kept working because somebody had a printout of all the flights. So a robust program keeps working even when the power is out. Though I think Heathrow has since “upgraded” to a new system that no longer works so well when the computers are down.

The Mars Orbiter crashed because one team used metric units and another team used awesome units. And mostly, because they failed to communicate. But really, because some code was reused. The original library, written for a previous mission, generated some data for a log file. Then for this mission it was reused and the log data was fed into the navigational system. Which effectively upgraded a not critical component to really critical status, but that change kind of slipped through.

Throughout the interview, Peter returns to the idea that software only needs to be good enough. If it’s 90 percent, and you ship something, that provides more benefit to more people than waiting until it’s 100 percent and delivering 0 in the mean time. And then you can go on to other projects. It might be possible to bring ten projects up to 90 percent in the time it takes one project to reach 100. But to return to the point that most software today consists of gluing together other software, I’ll do some math and note that 90 percent to the 10th is only 35% done. Which pretty much matches my observation of real world software.

founders

Paul Graham put Lisp on the web. Actually, he started trying to put art galleries online, but art galleries weren’t interested in being online. But then they switched to online storefronts, which people were interested in.

In the beginning they were desperate for users, because everybody could see how many users they had. An interesting hurdle for web software. People take one look and decide to pass because nobody else is using it, and so a lot of effort that could make the product better is instead spent attracting users. With downloadable software, nobody can see the userbase, so they’re more likely to judge the software on its merits.

Eventually they were bought by Yahoo and integrated. Paul’s analogy is that they were integrated the same way that a sugar cube is integrated into a cup of tea. The separate identity of Viaweb was completely dissolved.

Viaweb also had a power outage. But instead of using printouts, they bought a generator. It was too noisy to run in the office, though they tried, so they put it on the sidewalk outside and ran extension cords into the office. That is also a way to keep running when the power is out.

Joshua Schachter started delicious. A history of the past ten years of delicious would make for a very interesting read, but today we’re just here to talk about the early days. A lot like Yahoo itself, the early predecessor to delicious was just a list of links being personally maintained by Josh. As he describes it, it was single player. Then after noticing he had thousands of subscribers, he started adding multiplayer mode.

Because it was a small personal project to start, and he only worked on it in his spare time, he managed the codebase by keeping every function to one screen of code. That way, he could sit down and improve something without rereading the entire codebase. Just make a series of small isolated changes over the course of a few years and gradually things get better.

man-month

How much does a large program cost? Not to build, but to run? If you’re renting memory for $12 per kilobyte per month, running a large program can be quite expensive. On the other hand, features require space. We as users can’t demand software that does more without providing it with more resources.

A good program manager will establish some limits for size, but beware false economies. It’s easy to reduce size by shedding responsibility, but eventually somebody will have to take up the task.

The particulars of this chapter are somewhat dated, but I think some of the concepts are readily adapted to today. Or could be. Alas, they rarely are. How often does a program manager estimate how much code should be required to complete a task before development begins and then try to hit that target? More often it seems we throw code together until the solution is complete, and then wonder how we end up with gigabyte sized chat clients. Without an established target, it’s hard to identify where things went wrong. Perhaps everywhere.

pragmatic

23. Use asserts. Asserts are not to be used in place of regular, expected error handling, but to make sure the program hasn’t gone wildly off the rails. And definitely don’t turn asserts off in production. By definition, the assertion is checking for something unexpected, so it follows that you don’t have a test case for it. The unexpected conditions will occur unexpectedly! And then you’ll be happy the assertions are still there.

24. Use exceptions sparingly.

25. Finish what you start. If a function allocates a resource, it should free the resource. Also, free resources in the opposite order that you acquire them. This makes it easy to check for correctness. My experience has been that this approach solves a lot of problems, but code doesn’t usually evolve this way naturally. It’s only after a few bugs are found and fixed that I pay attention to object lifetimes and attempt to impose some sort of order. One final tip: since you’re bound to lose track of things, keep lots of counters. When things go mysteriously wrong, some examination of the outstanding object population may provide a hint.

code

We’re going to string some relays and gates together in surprising ways this week. First, what happens if we take the output of a relay and connect it to the input of the same relay? We get a flippy floppy. No, actually we get an oscillator. Some people like to call the oscillator a clock.

We might also wire up some NOR gates where the output of the first leads to the second, and the output of the second leads back to the first. Now this really is a flip flop. The output will be consistently on or off, as if it remembers what’s happening. If we want the flip flop to only remember inputs at certain times, we can add a Hold That Bit input. So now we’ve got a level triggered D-type flip flop. We might also call the Hold That Bit input the Clock. And we might call the circuit a latch. Stick a bunch of these together, and we have some memory for our adding machine. We can keep inputting new numbers and adding them to the existing sum.

And then we can make our flip flop edge triggered. And then feed it back into itself yet again, giving us a frequency divider. And then we combine a bunch of those together and we have a ripple counter. All sorts of nifty circuits are possible by feeding outputs back into the input. How about an edge triggered D-type flip flop with preset and clear? Sure thing.

It’s kind of hard to summarize a chapter that consists mostly of circuit diagrams, but lots of terms and concepts introduced here are starting to sound familiar. Latches and clocks and triggers sound like things that people writing operating system device drivers that interface with hardware care about. I’ve also found it occasionally helpful to reuse circuit concepts when building software state machines.

coda

When a program, or more precisely a component, is small enough we can understand everything it does. When it gets larger, we have to settle for understanding its interface. However, there can be two problems. First, the implementation may be incomplete or unreliable even though everything looks good from the outside. Second, we may also misunderstand the interface. The cumulative potential for problems increases with scale.

New package, tarsnapper-0.4.0OpenBSD packages
tarsnap wrapper which automatically expires backups

August 11, 2017

Fix for Ryzen usersDragonFly BSD Digest

If you are running DragonFly on a Ryzen CPU, this commit will fix (work around?) a hardware bug.  I have not looked at how other operating systems may be addressing it, but it may be interesting to contrast.

BSDCam Trip ReportMichael Lucas

The FreeBSD Foundation has posted my BSDCam trip report. I also want to sincerely thank the Foundation for helping defray my expenses.

Robert Watson runs an excellent event. And what I learned at BSDCam will appear through the new Absolute FreeBSD.

Speaking of which, I need to go make some words. Bye.

New package, buku-3.2OpenBSD packages
powerful command-line bookmark manager

August 10, 2017

a repo upon the deepTed Unangst (tedu@)

In reference to arbitrary code execution in various source control programs. Refer svn advisory. Remember A Fire Upon the Deep?

There’s some code archaeologists who dig up an artifact. They don’t know what it does, but it includes some instructions for how to unpack it. And so they follow the instructions. And they think they’re taking precautions to prevent it from doing bad stuff, but they screw up, and the evil AI is turned loose. And then bad stuff happens.

It’s funny how similar this is to today’s vulnerability. In theory, checking out a code repo should be a safe operation. All you’re doing is downloading some artifact from a server. Building the code, running the code, all that can be unsafe. But surely there’s no trouble to simply checking out some code?

Alas, a repo is not just a repo. Checking out a repo might require checking out other sub repos and external resources. And so a dumb read only artifact is actually a smart read/execute artifact. The artifact can’t be checked out without also interpreting some of its contents. And if interpreting happens to execute some unwanted shell commands... Bad stuff happens.

It’s a bug, and it’s fixed, but another lesson that nothing is ever simple when adding features. What looks like just a hostname over here could be interpreted as a shell command over there.

BSDNow 206: To hier is UNIXDragonFly BSD Digest

You all have seen the hier(archy) man page, correct?  BSDNow 206 gets into things like the new Lumina and Plasma desktops.

New package, virt-what-1.18OpenBSD packages
detect if we are running in a virtual machine
To hier is UNIX | BSD Now 206BSD Now
Lumina Desktop 1.3 is out, we show you a Plasma 5 on FreeBSD tutorial, explore randomness & more!
New package, migmix-20150712OpenBSD packages
mixture of M+ and IPA Gothic fonts, focused on kanji
New package, migu-20150712OpenBSD packages
mixture of M+ and IPA Gothic fonts, focused on kanji

August 09, 2017

New package, orthanc-1.3.0OpenBSD packages
RESTful DICOM server for healthcare and medical research
New package, py-pykwalify-1.6.0OpenBSD packages
lib/cli for JSON/YAML schema validation
Two bugfixes: sticky dirs and procfsDragonFly BSD Digest

Matthew Dillon has committed two bugfixes both to DragonFly-current and DragonFly 4.8.  I don’t know the exact circumstances under which it would affect someone, but the commits are there to evaluate.

openbsd changes of note 626Ted Unangst (tedu@)

Hackerthon is imminent.

There are two signals one can receive after accessing invalid memory, SIGBUS and SIGSEGV. Nobody seems to know what the difference is or should be, although some theories have been unearthed. Make some attempt to be slightly more consistent and predictable in OpenBSD.

Introduces jiffies in an effort to appease our penguin oppressors.

Clarify that IP.OF.UPSTREAM.RESOLVER is not actually the hostname of a server you can use.

Switch acpibat to use _BIX before _BIF, which means you might see discharge cycle counts, too.

Assorted clang compatibility. clang uses -Oz to mean optimize for size and -Os for something else, so make gcc accept -Oz so all makefiles can be the same. Adjust some hardlinks. Make sure we build gcc with gcc.

The SSL_check_private_key function is a lie.

Switch the amd64 and i386 compiler to clang and see what happens.

We are moving towards using wscons (wstpad) as the driver for touchpads.

Dancing with the stars, er, NET_LOCK().

clang emits lots of warnings. Fix some of them. Turn off a bunch of clang builtins because we have a strong preference that code use our libc versions. Some other changes because clang is not gcc.

Among other curiosities, static variables in the special .openbsd.randomdata are sometimes assumed to be all zero, leading the clang optimizer to eliminate reads of such variables.

Some more pledge rules for sed. If the script doesn’t require opening new files, don’t let it.

Backport a bajillion fixes to stable. Release errata.

RFC 1885 was obsoleted nearly 20 years ago by RFC 2463 which was obsoleted over 10 years ago by RFC 4443. We are probably not going back.

Update libexpat to 2.2.3.

vmm: support more than 3855MB guest memory.

Merge libdrm 2.4.82.

Disable SSE optimizations on i386/amd64 for SlowBcopy. It is supposed to be slow. Prevents crashes when talking to memory mapped video memory in a hypervisor.