Git

36 Articles

Advanced Techniques For Using Git With KiCAD

September 19, 2018 by 4 Comments

For most developers “distributed version control” probably means git. But by itself git doesn’t work very well with binary files such as images, zip files and the like because git doesn’t know how to make sense of the structure of an arbitrary blobs of bytes. So when trying to figure out how to track changes in design files created by most EDA tools git doesn’t get the nod and designers can be trapped in SVN hell. It turns out though KiCAD’s design files may not have obvious extensions like .txt, they are fundamentally text files (you might know that if you’ve ever tried to work around some of KiCAD’s limitations). And with a few tweaks from [jean-noël]’s guide you’ll be diffing and merging your .pro’s and .sch’s with aplomb.

There are a couple sections to the document (which is really meant as an on boarding to another tool, which we’ve gotten to in another post). The first chunk describes which files should be tracked by the repo and which the .gitignore can be configured to avoid. If that didn’t make any sense it’s worth the time learning how to keep a clean repo with the magic .gitignore file, which git will look for to see if there are any file types or paths it should avoid staging.

The second section describes how you can use two nifty git features, cleaning and smudging, to dynamically modify files as they are checked in and out of the repo. [jean-noël]’s observation is that certain files get touched by KiCAD even if there are no user facing changes, which can clutter patch sets with irrelevant changes. His suggested filters prevent this by stripping those changes out as files get checked in. Pretty slick.

Visual Schematic Diffs In KiCAD Help Find Changes

September 19, 2018 by 11 Comments

When writing software a key part of the development workflow is looking at changes between files. With version control systems this process can get pretty advanced, letting you see changes between arbitrary files and slices in time. Tooling exists to do this visually in the world of EDA tools but it hasn’t really trickled all the way down to the free hobbyist level yet. But thanks to open and well understood file formats [jean-noël] has written plotgitsch to do it for KiCAD.

In the high(er)-end world of EDA tools like OrCAD and Altium there is a tight integration between the version control system and the design tools, with the VCS is sold as a product to improve the design workflow. But KiCAD doesn’t try to force a version control system on the user so it doesn’t really make sense to bake VCS related tools in directly. You can manage changes in KiCAD projects with git but as [jean-noël] notes reading Git’s textual description of changed X/Y coordinates and paths to library files is much more useful for a computer than for a human. It basically sucks to use. What you really need is a diff tool that can show the user what changed between two versions instead of describe it. And that’s what plotgitsch provides.

plotgitsch’s core function is to generate images of a KiCAD project at arbitrary Git revisions. After that there are two ways to view the output. One is to generate images of each version which can be fed into a generic visual diff tool (UNIX philosophy anyone?). The documentation has an example script to help facilitate setting this up. The other way generates a color coded image in plotgitsch itself and opens it in the user’s viewer of choice. It may not be integrated into the EDA but we’ll take one click visual diffs any day!

Keep It Close: A Private Git Server Crash Course

June 27, 2018 by 65 Comments

At this point, everyone has already heard that Microsoft is buying GitHub. Acquisitions of this scale take time, but most expect everything to be official by 2019. The general opinion online seems to be one of unease, and rightfully so. Even if we ignore Microsoft’s history of shady practices, there’s always an element of unease when somebody new takes over something you love. Sometimes it ends up being beneficial, the beginning of a new and better era. But sometimes…

Let’s not dwell on what might become of GitHub. While GitHub is the most popular web-based interface for Git, it’s not the only one. For example GitLab, a fully open source competitor to GitHub, is reporting record numbers of new repositories being created after word of the Microsoft buyout was confirmed. But even GitLab, while certainly worth checking out in these uncertain times, might be more than you strictly need.

Let’s be realistic. Most of the software projects hackers work on don’t need even half the features that GitHub/GitLab offer. Whether you’ve simply got a private project you want to maintain revisions of, or you’re working with a small group collaboratively in a hackerspace setting, you don’t need anything that isn’t already provided by the core Git software.

Let’s take a look at how quickly and easily you can setup a private Git server for you and your colleagues without having to worry about Microsoft (or anyone else) having their fingers around your code.

Continue reading “Keep It Close: A Private Git Server Crash Course” →

Stupid Git Tricks

May 23, 2017 by 25 Comments

My apologies if you speak the Queen’s English since that title probably has a whole different meaning to you than I intended. In fact, I’m talking about Git, the version control system. Last time I talked about how the program came to be and offered you a few tutorials. If you are a dyed-in-the-wool software developer, you probably don’t need to be convinced to use Git. But even if you aren’t, there are a lot of things you can do with Git that don’t fit the usual mold.

Continue reading “Stupid Git Tricks” →

History Of Git

May 11, 2017 by 89 Comments

Git is one of those tools that is so simple to use, that you often don’t learn a lot of nuance to it. You wind up cloning a repository from the Internet and that’s about it. If you make changes, maybe you track them and if you are really polite you might create a pull request to give back to the project. But there’s a lot more you can do. For example, did you know that Git can track collaborative Word documents? Or manage your startup files across multiple Linux boxes?

Git belongs to a family of software products that do revision (or version) control. The idea is that you can develop software (for example) and keep track of each revision. Good systems have provisions for allowing multiple people to work on a project at one time. There is also usually some way to split a project into different parts. For example, you might split off to develop a version of the product for a different market or to try an experimental feature without breaking the normal development. In some cases, you’ll eventually bring that split back into the main line.

Although in the next installment, I’ll give you some odd uses for Git you might find useful, this post is mostly the story of how Git came to be. Open source development is known for flame wars and there’s at least a few in this tale. And in true hacker fashion, the hero of the story decides he doesn’t like the tools he’s using so… well, what would you do?

Continue reading “History Of Git” →

Git Shell Bypass, Less Is More

May 10, 2017 by 4 Comments

We’ve always been a fans of wargames. Not the movie (well, also the movie) but I’m referring to hacking wargames. There are several formats but usually you have access to an initial shell account somewhere, which is level0, and you have to exploit some flaw in the system to manage to get level1 permissions and so forth. Almost always there’s a level where you have to exploit a legitimate binary (with some shady permissions) that does more than what the regular user thinks.

In the case of CVE-2017-8386, less is more.

[Timo Schmid] details how the git-shell, a restricted shell meant to be used as the upstream peer in a git remote session over a ssh tunnel, can be abused in order to achieve arbitrary file read, directory listing and somewhat restricted file write. The git-shell basic idea is to restrict the allowed commands in an ssh session to the ones required by git (git-receive-pack, git-upload-pack, git-upload-archive). The researcher realized he could pass parameters to these commands, like the flag –help:

$ ssh git@remoteserver "git-receive-pack '--help'"

GIT-RECEIVE-PACK(1)            Git Manual             GIT-RECEIVE-PACK(1)

NAME
 git-receive-pack - Receive what is pushed into the repository
[...]

What the flag does is make the git command open the man page of git, which is passed on to a pager program, usually less. And this is where it get interesting. The less command, if running interactively, can do several things you would expect like searching for text, go to a line number, scroll down and so on. What it can also do is open a new file (:e), save the input to a file (s) and execute commands (!). To make it run interactively, you have to force the allocation of a PTY in ssh like so:

$ ssh -t git@remoteserver "git-receive-pack '--help'"

GIT-RECEIVE-PACK(1) Git Manual GIT-RECEIVE-PACK(1)

NAME
 git-receive-pack - Receive what is pushed into the repository

 Manual page git-receive-pack(1) line 1 (press h for help or q to quit)
 

Press h for help and have fun. One caveat is that usual installations the code execution will not really execute arbitrary commands, since the current running login shell is the git-shell, restricted to only some white listed commands. There are, however, certain configurations where this might happen, such as maintaining bash or sh as a login shell and limit the user in ways that they can only use git (such as in shared environments without root access). You can see such example here.

The quickest solution seems to be to enable the no-pty flag server-side, in the sshd configuration. This prevents clients from requesting a PTY so less won’t run in an interactive mode.

$ man less

LESS(1) General Commands Manual LESS(1)

NAME
less - opposite of more

Ironic, isn’t it?

Fail Of The Week: GitLab Goes Down

February 2, 2017 by 57 Comments

Has work been a little stressful this week, are things getting you down? Spare a thought for an unnamed sysadmin at the GitHub-alike startup GitLab, who early yesterday performed a deletion task on a PostgreSQL database in response to some problems they were having in the wake of an attack by spammers. Unfortunately due to a command line error he ran the deletion on one of the databases behind the company’s main service, forcing it to be taken down. By the time the deletion was stopped, only 4.5 Gb of the 300 Gb trove of data remained.

Reading their log of the incident the scale of the disaster unfolds, and we can’t help wincing at the phrase “out of 5 backup/replication techniques deployed none are working reliably or set up in the first place“. In the end they were able to restore most of the data from a staging server, but at the cost of a lost six hours of issues and merge requests. Fortunately for them their git repositories were not affected.

For 707 GitLab users then there has been a small amount of lost data, the entire web service was down for a while, and the incident has gained them more publicity in a day than their marketing department could have achieved in a year. The post-mortem document makes for a fascinating read, and will probably leave more than one reader nervously thinking about the integrity of whichever services they are responsible for. We have to hand it to them for being so open about it all and for admitting a failure of their whole company for its backup failures rather than heaping blame on one employee. In many companies it would all have been swept under the carpet. We suspect that GitLab’s data will be shepherded with much more care henceforth.

We trust an increasing amount of our assets to online providers these days, and this tale highlights some of the hazards inherent in placing absolute trust in them. GitLab had moved from a cloud provider to their own data centre, though whether or not this incident would have been any less harmful wherever it was hosted is up for debate. Perhaps it’s a timely reminder to us all: keep your own backups, and most importantly: test them to ensure they work.

Thanks [Jack Laidlaw] for the tip.

Rack server image: Trique303 [CC BY-SA 4.0], via Wikimedia Commons.