June 26, 2017

Removed, terraform-0.6.16Removed OpenBSD packages
tool for building, changing, and versioning infrastructure

June 25, 2017

In Other BSDs for 2017/06/25DragonFly BSD Digest

Back to overflow, which sort of makes my life easier.

Removed, mrxvt-0.5.4Removed OpenBSD packages
multi-tabbed terminal emulator

June 24, 2017

Ubiquiti Unifi and DragonFlyDragonFly BSD Digest

Should you need have Ubiquiti devices, and you need to get the Unifi management program running on your DragonFly server, this script will work for you.  Some of the filepaths are different, but it’s otherwise complete.

'Stack Clash' Linux Flaw Enables Root Access. Patch NowSlashdot
msm1267 writes: Linux, BSD, Solaris and other open source systems are vulnerable to a local privilege escalation vulnerability known as Stack Clash that allows an attacker to execute code at root. Major Linux and open source distributors made patches available Monday, and systems running Linux, OpenBSD, NetBSD, FreeBSD or Solaris on i386 or amd64 hardware should be updated soon. The risk presented by this flaw, CVE-2017-1000364, becomes elevated especially if attackers are already present on a vulnerable system. They would now be able to chain this vulnerability with other critical issues, including the recently addressed Sudo vulnerability, and then run arbitrary code with the highest privileges, said researchers at Qualys who discovered the vulnerability.

Read more of this story at Slashdot.

Removed, surf2-0.6.20160922Removed OpenBSD packages
simple webbrowser based on webkit/gtk+

June 23, 2017

WiFi232 with a Macintosh 512keJoshua Stein (jcs@)

Back in 2015, I created a BBS for Lobsters that worked in a web browser via WebSockets. After getting an old Mac earlier this year, I wanted a way to access the BBS from the Mac as natively as I could. Adding telnet and SSH frontends to the BBS was not too difficult, but being able to login from my Mac took a bit of work.

Macintosh 512k Enhanced

In January I got a Macintosh 512ke off of eBay and spent some time fixing it up. The screen would occasionally flicker and shut off, but banging on the side of the case would sometimes bring it back. Some research pointed me to the analog board needing some capacitors replaced, which has completely solved the problem.

The Mac had obviously come from a smoker's house so I was hesitant to even store it in my office because it smelled so bad. I took everything apart including the keyboard and mouse, scrubbed everything that wasn't electronic, and used zeolite rocks to remove the smell. I did a light retr0bright treatment on everything, though these early Apple computers originally came in a beige color before Apple switched to a more greyish "platinum" color, so the restoration was not very dramatic.

Communicating With a 30-Year-Old Computer

Instead of constantly swapping floppy disks, I got the wonderful Floppy Emu that plugs into the drive port on the back of the Mac and set it to emulate an HD20 hard drive. This also makes it relatively easy to transfer data between the Mac 512ke and my MacBook by swapping the Floppy Emu's microSD card into my MacBook and using Mini vMac on macOS to boot a virtualized Mac environment off the HD20 disk image on the SD card.

With MacTerminal installed, I set up the Mac as a serial terminal for an OpenBSD computer running on a BeagleBone Black. This took some time to figure out because these early Macs use RS422 serial ports instead of RS232, so I had to make a custom pin conversion between the Mac and the BBB's serial port.

Once connected to OpenBSD, I was able to SSH into the Lobsters BBS, but I wanted a more authentic experience for the Mac.

A Detour Into LocalTalk

As memory constrained as the Mac 512ke is, it is supposedly capable of doing native TCP/IP over Ethernet. I found an Asante Talk bridge on eBay and tried setting up TOPS Terminal and its TOPS TCP/IP stack. Unfortunately I was never able to get this working as I could never get the Asante Talk device to send any AppleTalk traffic to my network.

Despite being "native" TCP/IP, this type of setup needs a lot of external components, not the least of which is a separate AppleTalk gateway that has to be running on a local network server to unencapsulate the TCP/IP traffic.

WiFi232

Soon after being defeated by LocalTalk, I discovered the excellent WiFi232 which is a small WiFi-enabled board with an RS232 DB25 serial interface that emulates a Hayes-compatible modem. Instead of using ATDT 5551212 to dial a phone number through a modem, the WiFi232 allows one to do ATDT lobste.rs and establish a telnet connection over TCP/IP. The computer (and any associated terminal/modem software) thinks it's talking to a serial modem and the WiFi232 does all of the legwork of connecting to WiFi, getting an IP via DHCP, and doing telnet negotiation over TCP/IP. It even has an AT-compatible settings menu, firmware upgrades, and a web server for more configuration, though I never bothered to set that up.

The WiFi232 can do proper telnet negotiation with the ATNET1 configuration setting, and negotiates a TTYPE of ANSI. MacTerminal doesn't support the 437 codepage, so most BBS line graphics look wrong, though I did transfer over an old copy of Z-Term which does support 437. Unfortunately since the Mac's monitor is black and white, ANSI colors don't gracefully degrade so most large graphic areas (like the Lobsters BBS welcome screen) look solid black. I was able to fix some telnet negotiation bugs in my telnet daemon that I came across while playing with the WiFi232, but due to the limitation of the Mac's screen, I've disabled telnet negotiation for now with ATNET0 so I get the ASCII version of the Lobsters BBS. Perhaps I'll add an ANSI-without-color version to Lobsters after prompting the user if they want color, like many other BBS packages do.

One configuration setting I wish the WiFi232 did have was a way to keep telnet negotiation enabled but change the TTYPE that it reports. The WiFi232 creator was quick to implement being able to set a new TTYPE with a new AT$TTY=n setting. Highlighting another great feature of the WiFi232: updating to a new firmware is as easy as just issuing an ATUPDATE command.

Using the WiFi232 with the Mac still required my RS422-to-RS232 wiring mess, and now also required a DB9-to-DB25 adapter. The whole contraption was getting a bit messy and eventually I wanted a more elegant solution.

Apple Modem 300/1200

For the early compact Macs and the ][, Apple made an external 300/1200 baud modem. Eventually I found one on eBay but it was in rough shape and had no AC adapter, though it did include a genuine Apple serial cable. Since the modem is pretty long (to accommodate a telephone placed on top of it), I thought it would make a nice housing for my Wifi232 and rats nest of wires.

Unlike my Mac, the modem was very yellowed and a couple retr0bright sessions lightened it up quite a bit. Unfortunately I left it outside in the sun a bit too long without moving it, so there appears to be a bit of bleaching in some areas.

Before and after retr0brighting with 40-volume developer creme

Initially I had planned on removing the modem's entire circuit board, but I wanted to retain the original phone jacks and other ports on the back even if they weren't going to be used. Since these ports are attached to the circuit board and the board is held into place through the screw holes in the case, I would have had to make a new board out of metal or wood, cut it to size, de-solder all of the components from the circuit board, and then attach them to the new platform in the precise location to fit in the slots in the rear of the case.

Since the modem is not very useful as vintage hardware without an AC adapter and is limited to 1200 baud anyway, I decided just to cut the circuit board to be able to retain the rear ports and still give room for the WiFi232. While the modem originally had a strange 4-pin AC adapter connector, the WiFi232 uses mini-USB for power so I added a mini-USB extension cable and glued it in place of the original port.

The WiFi232 installed, the modem's new rear ports, and USB power and serial cables

The original modem board had a green LED that shined through the front case when powered, but I had cut off that section of the board. I soldered some leads to the WiFi232's power pads and then hooked up a new green LED and resistor to them to restore the power light.

The modem also had a toggle switch in addition to the AC port, and in the future I'd like to wire in a new toggle switch to kill power from the micro-USB cable. Right now I have to remove the micro-USB cable to power down the modem each time. The Mac actually sends 12V of power through one of the RS422 port pins so it might be possible to just power the WiFi232 off it whenever the Mac is turned on, but for some silly reason that seems less genuine.

Now that the WiFi232 is completely enclosed in the Apple modem case and only has a micro-USB power cable and Apple serial cable coming out of it, I think it fits in nicely with my Mac.

I am planning on getting a Western Electric 2500 telephone to put on top of the modem like it was designed to accommodate. Maybe I'll put a small VoIP ATA inside of it so it can function like my payphone.

books chapter oneTed Unangst (tedu@)

I wanted to read, or reread, some books, but couldn’t decide which ones, so figured reading all of them at once would be the best solution. In particular, I’d read Coders at Work about the time it came out, and liked it, then skimmed it again recently. The second time through I still liked it, but I noticed new things. I should reread the whole thing. And what about these other books I’m always certain to install on each Kindle but never quite read? My favorite unread books.

There’s Coders at Work, 15 interviews with people who code. There’s the book that inspired it, Founders at Work, which also contains some good development advice along with the business ideas. That’s 32 chapters, so double time through this one. There’s the Mythical Man-Month, 15 or 18 or so chapters. The Pragmatic Programmer has 8 chapters, of a great many sections each, so that can be read at half speed. At some point in the past I read most of Programming Pearls and liked it, but don’t have a copy at hand, so instead the last selection is something a little different, Code: The Hidden Language of Computer Hardware and Software. 25 chapters, start with two per week and slow down for the hard stuff. That’s five books, which is probably pushing the limit of my ability to consume words requiring thought. At least the chapters for most books are independent, so I won’t have to remember anything.

coders

Our first victim is Jamie Zawinski, who wrote a text editor and a browser. But before all that, he was working in Lisp, which is a lovely language. jwz also has opinions about C, and perl, and something called C++. While at Lucid, building an IDE out of emacs, he was in charge of vi emulation. “The real problem with that wasn’t so much that it was emulating vi wrong as that vi users quit and restart vi all the time. And no amount of coding on my part is going to get them out of that mindset.” This is a true observation.

The early Netscape team was only one or two people per project: networking, layout, platform specific front end. They got a lot done in only six months, but basically only by working non stop. And rushing to get things done meant some of the quality wasn’t top notch. But there’s no time to rewrite because you’re immediately on to the next thing. All of which sounds about right. You can get a lot done working independently. Though I’ve worked some places where the motto was closer to there’s no time to do it right, but there’s always time to do it over. I guess if you don’t even have time to do it over, you’re really going to feel the pinch.

And then Collabra bought Netscape. No, wait, other way around, but Collabra ate Netscape from within and ruined everything by using threads. And also, compromising the cross platform portability by releasing Windows first, and then porting later. Mixed thoughts on this. Developing for one platform and then porting to another has resulted in some pretty compromised results. A lot of little assumptions sneak into the code and it takes a long time to fix, and some of the assumptions don’t get noticed and then you ship something broken. In many ways, portability is an attribute a lot like being able to compile. If your code doesn’t compile, or doesn’t work, but you keep plowing ahead bad things will happen. Similarly with disregarding portability to get things done. On the other hand, OpenSSH is developed for OpenBSD first, then ported, and I think the results turn out ok. Sometimes you just want to look at the code for one platform to see how things are supposed to work without getting tied down in all the reality. jwz also mentions porting XScreenSaver to OS X by reimplementing Xlib in terms of Cocoa, so he didn’t have to change the main code base. This approach often works really well. Pick one platform, then make the other platforms be that platform. Your code is always portable, but it’s always concrete, in terms of a known platform, not an abstraction layer that exists nowhere else.

“Usually by the second or third time you’ve cut and pasted that piece of code it’s like, alright, time to stop cutting and pasting and put it in a subroutine.” I like the rule of three most of the time. It’s ok to duplicate code because you don’t know yet what the common elements should be. Once you’ve done something three times, you know what it is you’re doing.

There’s also a lot more talk about rewriting. It’s fun to write new code, to do it the right way, to replace this thing somebody else wrote which you’d have to study to understand. But the result is nothing is ever quite done. The year of the Linux desktop will be next year, for sure.

founders

Max Levchin founded PayPal. An interesting tale of continuously pivoting and refocusing the product. He started out writing a crypto library. But nobody was much interested in writing software, or at least not software that needed a secure crypto library. So he switched to writing such software, a password keeper. But users didn’t seem to care much about securing their bank password, either. So he switched to storing money directly. Now suddenly users were interested. You could securely beam crypto money from one Palm Pilot to another. And then, almost as an afterthought, to show people how cool life would be carrying a crypto wallet on your Palm Pilot, they built a web interface that let people without PDAs send money. And boom, people liked this. A lot. The crypto money of the future was quickly killed in favor of boring old web banking. What I found interesting is how he had to keep climbing up the stack, in theory shrinking his market. The crypto library he started with could have been used not just by PayPal, but also by their competitors, and the password keepers, and everybody else. That would be a much bigger market. But nobody was interested in his shovels, so he had to go out and do the digging himself.

In the early days, advisors told him he’d be buried by fraud. Fraud? What fraud? I don’t see any fraud. Until you wake up one day, the day after somebody has figured out how to do automated chargebacks, and boom. You’ve got fraud. And now you’re in fire fighting mode, trying to solve this problem in real time even as it keeps getting worse. I liked two things about this story. Sometimes insiders really do have some domain specific knowledge about what makes things harder than they seem. And things can go from no problem at all to full on conflagration pretty quickly. On the other hand, he says that one reason the banks took so long to tackle personal payments is their zero risk attitude. They knew there would be fraud, and they weren’t willing to bleed money until they solved it. Sometimes a problem isn’t solved except by someone who encounters it too late and has no choice, while everyone else does their best to avoid it entirely.

On the importance of reporting and visualization. Early fraud fighting was done by hand, by reviewing transactions. It wasn’t effective. So they built some special tooling, which did two things. It let them see what was happening, where the money was going. And it predicted and prioritized what were likely to be the largest losses, which wasn’t possible just from looking at thousands of pages of printouts.

Sabeer Bhatia founded Hotmail. It was supposed to be a different company, but he couldn’t send personal email to his cofounder through the corporate firewall. A web gateway to email solved that problem. And then it turned out to be convenient for other people, too.

He has some theories about software development. He comes from a hardware background, where you have to get things right the first time because it’s very expensive to fix afterwards. No patching. And thus they make better software developers because they’re more methodical and understand state machines and so forth. I’m not sure I agree with this; the perception in my circles is that hardware engineers write the worst software (usually monstrous piles of poo pretending to be device drivers). But of course, there could be a difference between a hardware engineer temporarily forced to write a driver before getting back to the fun stuff, and a hardware engineer turning to software development full time. His point about the approach one takes is a good one. When you model software as a deterministic function, that takes some inputs and produces some outputs, that leads to a better result. The ship then patch mentality trends towards the lower end of the quality spectrum.

Crazy story: one person didn’t type hotmail.com into the location bar, but went to Yahoo and searched for Hotmail and clicked on that. Of course, people still do that today, though possibly with less Yahoo. I find it funny that he found this behavior unbelievable in 1997, a time when URLs were probably very new to many people. Getting to websites through a portal would have been pretty natural. And a great deal of Hotmail’s success was because they made it easy for unsophisticated users to use email. So don’t be surprised that users find a different way of doing things than you do.

“If you are expecting people to dramatically change the way they do things, it’s not going to happen. Try to make it such that it’s a small change, yet an important one.” A true observation. People are going to do things the way they like doing them, and they’re not going to stop using vi. Oh, wait, different interview.

man-month

Programming is a tar pit. The more you struggle, the slower you go and the more you’re sucked down. Nevertheless, each individual limb is free to move. What could be the problem? This is a great analogy. You can be drowning in tar for quite some time before you realize what’s happening, and then it’s often too late.

We begin by considering the two programmers in a garage that get things done. A classic myth that’s generally expressed today as “I could build that in a weekend.” Don’t even need a second programmer or a garage. But the output of our lonesome programmers is a mere program. It’s not a product, complete with documentation that doesn’t crash on unexpected input. That’s three times the effort. Nor is it a system, which is to say it’s not reusable, with standardized interfaces. There’s another three times effort. So all told, turning a one off program into something one might reasonably sell requires a nine fold increase in effort. Easier to start things than to finish them.

The joys and woes of programming. Programming is fun because we get to learn and build things. We can see our ideas take form. But then reality sets in. There’s always one more bug to fix, which isn’t very fun, and sometimes we have to depend on other people’s code, which is always the worst.

And then there’s the sense of dread, as a project nears completion, that it’s already obsolete because people are looking ahead to the next version. Or a competitor’s product, which promises to do more. Why even finish? One good reason is that next version is likely to be late, over promise, and under deliver. I liked this as a counterpoint to the sense of urgency expressed by others. In a sense, he’s saying the same thing. Something that ships has much more value than vapor. But everyone else is dealing with the same constraints and pressures.

pragmatic

1. You are responsible for your work, even if the cat eats your source code. Provide options, not excuses. Eh, meh.

2. Software is susceptible to entropy. Pay attention to how things are going, and fix up the broken things before they get worse. (Otherwise you end up being Netscape.) A reference is made to the broken windows theory. Somewhere on the OpenBSD web site it states “Do not let serious problems sit unsolved.” So this sounds like pretty good advice, though of course it’s subject to some interpretation, which is probably the hard part.

3. The parable of stone soup is related, wherein everybody contributes a little something, even if only reluctantly, and suddenly there’s a pot of delicious soup for all to enjoy. A bonus parable of the boiling frog is related as well. The foreword to this book promised that every tip was concrete advice drawn from experience. I suppose one could make stone soup from concrete blocks as well.

code

I’m pretty excited about this book, which is actually a new addition to the collection, and one I haven’t tried reading before.

The word code is often associated with computers and programming (not surprisingly that’s the common usage in all the previous books as well), but really it can be any system of transferring information. I want to send you a message by waving a flashlight, perhaps using Morse code. That’s a code. Codes can be arbitrary. Why are dogs called “dogs” and not “cats”? The important property of a code is that it be intelligible. So codes aren’t just in computers, they’re everywhere. Some codes may be arbitrary, but some, like Morse, are designed to be efficient. Common letters are assigned shorter sequences.

Speaking of Morse, apparently during WWII the BBC would play the Fifth Symphony opening, dih dih dih dah, because that’s V. For Victory. Of course, Beethoven didn’t intend to convey that meaning. The book glosses over this (at least for now), but I think it’s an important point. The reinterpretation and misinterpretation of codes can be very powerful. We can’t change what someone says, but we can change what they mean. And in reverse, of course. So that’s something to think about.

Key observation about Morse: dots and dashes. Two symbols. Two. Apparently this is important. Little foreshadowing.

Chapter two continues our investigation of Morse. The typical table maps letters to dots and dashes, which makes it easy to send. But decoding an incoming message requires scanning the second column of a table with no apparent ordering. Imagine how much easier it would be to use a reverse lookup table. While building our reverse table, we might notice that it can also be represented as a tree, with each node branching dot or dash. At the top we have E and T, dot and dash. Each level we add to the tree doubles the number of letters, plus numbers and symbols, we can represent. And if we extend things all the way to 8 levels, 8 possible dots or dashes, that may even be enough to represent a great many symbols.

coda

Take your time. Do things right. But don’t take too long. Get things done. Life sure is hard. Unless you’re Goldilocks and everything is just right.

A common theme does seem to be that it’s easy, in the moment, for the sake of expediency, to cut a corner. And maybe this is even the best thing to do for now. But such quick hacks have a tendency to pile up until they become the only way to get things done. So keep a weather eye out.

Code that doesn’t ship isn’t worth much. Heard that a couple times. But at what cost?

New package, waagent-2.2.13OpenBSD packages
Microsoft Azure Linux Agent
OpenVPN 2.3.17 on OpenBSD 6.0packetmischief.ca

On Jun 21, the OpenVPN team released an update for the 2.3.x and 2.4.x branches that resolved some newly discovered security vulnerabilities. The OpenVPN team recommends that users “upgrade to OpenVPN 2.4.3 or 2.3.17 as soon as possible“.

OpenBSD 6.0–which was released Sep 1 2016 and is still receiving security updates to the base system as per OpenBSD’s policy–shipped with a package for OpenVPN 2.3.11. Below you will find a patch and instructions for using the ports system to upgrade to version 2.3.11. Note that if you’re running OpenBSD 6.1, the ports tree has been updated to 2.4.3 so all you need to do is “cvs up” and “make install”.

Instructions:

  1. Follow the OpenBSD FAQ for instructions on how to download, verify, and extract the ports tree on your machine.
  2. Then:
% cd ports/net/openvpn
% patch < ~/openvpn-2.3.17p0.diff
% make install

Original article: OpenVPN 2.3.17 on OpenBSD 6.0

Copyright © 2017 Joel Knight . All Rights Reserved.

New package, py-iso3166-0.8OpenBSD packages
self-contained ISO 3166-1 country definitions library
New package, py-iso639-0.4.5OpenBSD packages
Python library for the ISO 639 language code standard

June 22, 2017

BSDNow 199: Read the source, KARLDragonFly BSD Digest

No interview in this week’s BSDNow, but there’s a good run through recent BSD news, including some talk about a “Aeronix” machine project, which led me to some other interesting links.

Tinc VPNLobsters

I’m posting a link to Tinc VPN because I am interested in comments about the security and usefulness of tinc. I have started using it for a few projects running on OpenBSD because it is very easy to get up and running and also reconnects quite easily should network conditions change which is something that isakmpd and iked are not always so good at. Any comments or concerns about its use?

Comments

Read the source, KARL | BSD Now 199BSD Now
FreeBSD 11.1-Beta1 is out, we discuss Kernel address randomized link (KARL), explore the benefits of daily OpenBSD source code reading & more!
OpenBSD now has Trapsleds to make life harder for ROPersUndeadly
You heard it here (or on tech@) first: Trapsleds are in, and it makes OpenBSD even safer. Work done by Todd Mortimer and submitted to tech@ in the Trapsleds thread was later committed by Theo de Raadt.

Todd's message to tech says,

I have attached a patch that converts NOP padding from the assembler into INT3 padding on amd64. The idea is to remove potentially conveinent NOP sleds from programs and libraries, which makes it harder for an attacker to hit any ROP gadgets or other instructions after a NOP sled.

Read more...

alloca with great difficultyTed Unangst (tedu@)

All the cool kids are clashing their stacks, and all the cool developers are trying to reduce stack usage. In the midst of this, it is revealed that calling alloca can be difficult.

For starters, we might look at this fine patch removing alloca from a function in glibc. I’m mostly interested in the first chunk. That’s quite the incantation to prototype a function.

Another variant of the alloca spellbook is in bash. This version supports a different set of operating systems.

As Ben Franklin never said, “Beer is proof God loves us and wants us to be happy.” The ifdef maze one encounters trying to call alloca is proof your compiler hates you and you will be unhappy.

June 21, 2017

ig4 support for newer CPUs on DragonFlyDragonFly BSD Digest

Thanks to Imre Vadasz, your Haswell, Atom SoCs, Skylake, and Kaby Lake CPUs will now recognize the ig4 device on the associated motherboard.   I think it does something with ACPI?  I have always been hazy on smbus device functions.

New package, pdal-1.5.0OpenBSD packages
translator library for point cloud formats
New package, py-laspy-1.5.0OpenBSD packages
library for reading, modifying and creating LAS LiDAR files

June 19, 2017

Free and Libre Open Source Software (FLOSS) @D2SIAntoine Jacoutot (ajacoutot@)

Zealots repellent

In this article I am going to take some liberty and refer to FLOSS (Free and Libre Open Source Software) as « open source » or « free software ».

 

Teh origins

A few months ago I was put in charge of developing an open source ${THING} at D2SI. THING was still undefined at the time but could have probably been found at the intersection of strategy, cell, offer, communication and vision. At first I was amazed that in 2017 companies still needed to be introduced to the FLOSS ecosystem. Anyway I thought it was an interesting idea and got very excited, very fast!

Oh boy, little I knew I’d be left alone naked in front of that task…

After a few days of thinking, I came to the conclusion that I had no clue what I was doing. I mean I’ve been contributing to and using open source software for about two decades but what the hell could a consulting company possibly offer to the community? We are not a software company. So what does it mean for us to have an open source ${THING}?

Sure we are using FLOSS on a daily basis but again, we’re not a development company, so it’s not like we have code we can open source… or have we? We spend countless hours everyday writing IaC (infrastructure as code). Could that be something worth sharing?

Fr33 7H3 C0D3

This almost immediately brought up the question of added value. As a consulting company, what do we sell? When I look at people spending their days on stack overflow and copy/pasting random things from the internet, I am convinced that our added value is not in our terraform recipes for example but in our experience and reasoning that led us to build things the way we do.

We don’t believe there is any negative impact in opening our code, recipes and development process. Plenty of individuals and organizations already do so. It’s not trivial to incorporate outside code and in general it’s much more beneficial for our customers to come to us to do the integration together because that’s where our expertise kicks in.

But let’s not kid anyone here, that approach is also highly beneficial to us. Because working in all kind of different setups will help our consultants to enhance the code and come up with something reusable and multipurpose (i.e. not tight to a particular environment). The goal is to spend more time on things that matter and are fun.

So by itself our IaC has no great interest. But shared amongst our pool of consultants and the community at large, there is a chance we can make it awesome for everyone…

OK, that could actually work. Let’s take it for a test drive! Of course IaC is just a poor excuse to kickstart the project but we have to begin somewhere. Talking to people around D2SI I came to realize that there are many other great things we can share with the community as we will see below.

Here be dragons!

Then there was the question of the license. What should we use by default in our projects? As most people familiar with FLOSS know, discussing this topic almost always ends up in a religious war.

Now, I’m a BSD guy and I don’t like viral copyleft licenses like the GPL. I much prefer the « no strings attached » approach of BSD-style licenses which basically says:

  • do whatever you want with the code, yeah that includes building a nuke
  • don’t complain if it does not work or kill your puppy
  • don’t pretend you wrote the code (no one would believe you anyway)

<troll>Yes I am aware I am just throwing a totally subjective opinion up in the air.</troll>

Anyway, because I am old and wise, it wasn’t hard to convince management to go for one of the most permissive licenses: the ISC license which is used by the Internet System Consortium (duh!) and most importantly OpenBSD.

 

Side note: if one ever wanted to understand some of the differences between open source and free software, looking at the 3.3 clause of the Amazon Software License would be a good start. Code distributed under this license may be open source but it’s not free software (usage limitation).

EPIC FAIL: GitHub is not free!

Yeah yeah yeah, GitHub is not free software, but like or not it won… for now. So let’s be pragmatic and put things where people expect them to be. Also everyone knows how to open a pull request and contribute to a GitHub project. We are trying to make it easy for anyone to do so or report issues.

As of today, here are a few projects hosted at https://github.com/d2si-oss :

  • data-engineering-coding-challenge
    Coding challenge for future D2SI data engineers
    This is a technical test for data engineering candidates. They should do this test on their own. It’s designed to take 2 or 3 hours but there is no hard limit.
  • demo-aws-lambda-buffer-api
    Deploy a serverless API Buffer using AWS Lambda and API Gateway with Terraform
    The basic requirements for the project are an HTTP endpoint that accepts a JSON GET/POST/PUT/DELETE/OPTIONS containing random data which will need to be stored somewhere. This required a few components:
    – a front-end to take in HTTP requests: API Gateway
    – a back-end for handling these requests: Lambda
    – a datastore to keep all the associated short tokens and API responses: DynamoDB
  • ooso
    Java library for running Serverless MapReduce jobs
    Ooso lets you run MapReduce jobs in a serverless way. It is based on managed cloud services: Amazon S3 and AWS Lambda and is mainly an alternative to standard ad-hoc querying and batch processing tools such as Hadoop and Spark.
  • terraform-modules
    Reusable Terraform modules
    This repository holds modules for the terraform IaC tool. They are arranged by providers type (e.g. aws, google…). The root of the repository contains sample stacks exercising some of these modules.

This last repository is still pretty young and small because it is meant to onboard D2SI consultants. You catch more consultants with honeyform than you do with CloudVinegar.

We also encourage our teammates to fork upstream projects they want to contribute to under the d2si-oss GitHub organization. That’s how we ended up contributing to a few emblematic projects like terraform or the aws-cli.

We’re just starting but we already have a few projects in mind like a training environment bootstrap recipe using packer, terraform and/or ansible for instance. We offer different kind of trainings via Icelab and there’s no question people would appreciate a bootstrapping kit to automate student requirements (instances, user account, pre-installed software etc.) by just defining a few variables.

We would also like to promote some technologies that we believe are better suited for some tasks than the industry would usually promote. OpenBSD being an example (multiple region / cloud providers connectivity over IPsec, SSH bastion, RDP proxy, DNS forwarder…).

Not everything has been published yet because we are still cleaning legacy code, anonymizing to make things are generic as possible (e.g. Ansible roles). Also some of our internal projects are still poorly documented and we consider bad documentation the same as bad code so we need to fix that before opening it.

SAMCRO

We realized eventually that building a team around a project like this one made perfect sense for a consulting company. Consultants spend most of their time at customers so it’s tricky to build a team spirit and a sense of belonging. Sure we’ve been doing loads of internal events so that people could meet, exchange and share but that is different. Working together on such a project really makes you feel like you’re part of the club and gives you a different sense of purpose that of your daily job. We like to see it as a job spin-off. Gathering people around a common goal matches the way most free software projects are developed with contributors spread around the globe.

I feel like I need to stress something now: this proposal is solely based on volunteering, no one is forced to join the effort. Also this is not a way for your employer to coerce you into doing extra work, actually quite the opposite: while chatting with fellow co-workers it was brought to my attention that more than a few of them already contributed in their own time on software we use in our daily job. We can make this a win-win by giving them time during work hours to contribute: they do what they like, it makes us look good and we become more attractive to new recruits. We could even offer fun rewards to our top contributors!

There are also obvious benefits to everyone: a few weeks ago we had the choice between implementing something ugly or extending terraform with a new feature to satisfy a specific deployment need. We made time for our terraform contributor to develop the feature which made our customer very happy. It’s been merged upstream since, having to maintain a local fork and a local build would make no sense whatsoever.

Some people also expressed the will to contribute to upstream projects but need help to do so. Not technical but emotional help because it’s not always easy for anyone to confront ideas, deal with rejection etc. We think we can accompany them in that regard the same way we encourage our staff to speak at meetups and conferences.

We are also looking into hosting hackathons to create reference implementations of recurring infrastructure tasks, agree upon our best practices and stop reinventing the wheel each time we start a new project.

Shut Up and Hack!

If it weren’t for open source and the free software movement, we would all be working with immensely boring closed source technologies. Well not me because I’d probably be doing something else… Not to say our daily work isn’t surrounded with closed and opaque black boxes (I’m looking at you Cloud providers) but as least most of the tooling, software and operating systems we use are open source and more importantly free (as in freedom).

Anyway we are trying to build our own ${THING}, learning as we go to ultimately have fun! It will take time but we are confident we can create something that will work out for all of us at D2SI. We are not afraid to show what we do, what we know what we don’t know, how we work… And we sometime even talk about it in our brand new technical blog: http://techblog.d2-si.eu/.

An open source project is all about the community. Without it, it’s just yet another code rot repository. While we hope to create a community within D2SI to begin with, it is our goal to broaden the scope. So have your pick of https://github.com/d2si-oss. We value external feedback as much as internal one and there’s no precedence whether you’re a consultant at D2SI or not: https://github.com/d2si-oss/contributing-guidelines.

After working towards corporate liberation to unleash the potential and initiative of our staff, liberating the code was a natural next step.

Cet article Free and Libre Open Source Software (FLOSS) @D2SI est apparu en premier sur D2SI Blog.