Building a Private Conda Channel

         ·      · ·

I first wrote this blog when I was working at City Science Why we need a private conda channel Here at City Science we build a large number of reusable Python modules. For developers to consume these libraries, we needed to host the different versions at a central location. There are public hosting solutions available, but for now we needed our libraries to remain private. For that reason we opted to create our own channel.

Tool Mastery


Whilst working with a new client which had some green developers, it struck me that they only knew about 20% (totally arbitrary for this example) of the tools features they were using on a daily basis. For example, when asking them if they preferred merge vs rebase, I was often met with blank stares. Similarly, when using their editor I would often find them manually searching and replacing renamed variables, rather than using the built-in feature of the editor.

Awesome Jenkins Pipelines

              · · ·

A curated list of awesome Jenkins reusable pipeline code. jenkinsci buildit Invoca Docker team Funkwerk Spring Cloud kitconcept SAP

Override Systemd Config


I was faced with the task of enabling the remote API for Docker. Whilst trying a few different solutions (adding hosts to daemon.json), I quickly learnt that I needed to pass parameters to the dockerd command like /usr/bin/dockerd -H tcp:// -H unix:///var/run/docker.sock. As I’m using Systemd I knew I needed to edit the service file, but I was also mindful that any future updates to Docker would blow my changes away.

Git reset last commit


Sometimes I get carried away with git and its high power features. I was working between 2 repos and realised I had a change I hadn’t saved and wanted to add to my last commit. This was my blog, of which im the only contributor, and so I made the save (resulting in an unstaged change) and run git add . && git commit --amend --no-edit. To those who dont know this command will stage all new changes and add them to the previous commit without making you edit the commit message.

SBT to Gradle


Here’s how I made the switch from sbt to gradle for a play project. Sbt works, there’s no denying that, but having spent many years working with Gradle I found it frustrating to replicate what was very simple with Gradle. My current clients play application is built with sbt; you can find a link to the build here. Here’s a snippet from build.sbt import play.sbt.PlayScala import sbtbuildinfo.BuildInfoPlugin.autoImport._ import sbtassembly.AssemblyPlugin.autoImport._ import com.

log tips and tricks

              · ·

As engineers we’re constantly trawling through logs. Here’s some tips for making that task a little easier. Searching logs For larger logs we may want to scroll by page: cat mylog.log | less We can then use PgUp or PgDn or the mac equivalent. Regex search in less /<regex> Find error with grep $ grep -I error /var/log/messages or find firewall related stuff sudo tail -f /var/log/messages |grep FIREWALL

Bulk find and copy in Bash


I needed to move a bunch of files from an old blog to the new one for conversion. Here’s how: find . -name \*.asciidoc -exec cp {} /home/sionwilliams_com/site/content/post \; From the current directory it searched for files ending .asciidoc and moved them to the new location. Once in the new location I was able to change the extension of the files with: for f in *.asciidoc; do mv -- "$f" "${f%.

Making Ticketing Systems Useful


As software engineers we’re all guilty of using ticketing systems as expensive todo lists. We transition tickets from todo to in progress to done, but rarely go beyond that functionality. We may have nice integrations which link back to our version control system, or open some dialogue with fellow developers, but thats often the limit of our usage. I’ve recently started using the ticketing system as my personal journal. In my opinion, a ticketing system should include a record of the trials and tribulations involved in delivering a feature/issue, not just the about our interpretation of the issue/bug.

Parallelise Vagrant Up With Make


Spinning up more than one Vagrant box can be slow and tedious. Below is a solution for speeding up the vagrant up process when you need more than one box and Docker isn’t a valid option. Annoyingly, this solution requires a Vagrantfile per machine, and wont work if you put loops in your Vagrantfile. Here’s my folder structure: . ├── Makefile └── vagrant ├── node-1 │ └── Vagrantfile ├── node-2 │ └── Vagrantfile └── node-3 └── Vagrantfile The Makefile is where the magic happens…