It's hard work running a business and keeping this site updated!
For the forseeable future I will be focusing all all my technical writing onto other platforms as traffic is much higher.
You can check there out in these spaces:

Twitter   Medium   Hackernoon   Coil  

October 1, 2020.

Medium article

Importing Existing Infrastructure into TerraformTerraform is a powerful tool to have in your toolset. It's effortless to use, for creating new infrastructure, but not so much for importing existing infrastructure,...

February 6, 2020.

Airflow dynamic DAG graph flow changes at run time.

I have recently been working on as project which involved Airflow, and the project required a dynamic workflow. The project went something along the lines of: Remove old files from...

February 1, 2020.

Performance issues with AWS Glue

I recently spent some time looking at an AWS Glue job which was not performing as expected. The job was taking a file from S3, some very basic mapping, and...

January 16, 2020.

Visualising AWS Glue history in Spark UI History Server

I recently had to try and spot why a job running in AWS Glue was not performing as expected. Glue outputs a huge amount of logging information, most of which...

December 1, 2019.

Diving into AWS Athena

Benefits of Athena Athena is serverless, which means there is less management and maintainance overhead to run day to day tasks, such as updates, failovers etc. It also means that...

November 29, 2019.

Example of an ETL job in AWS Glue, and query in AWS Athena

I hope with this post to show a simple end to end run through of using AWS Glue to transform data from a format, into a more queryable format, and...

November 21, 2019.

Overview of Data Analytics in AWS - Glue, Athena and DataLake

I hope with this post to discuss the current state of analytics in AWS Cloud. What is a data lake? A data lake could be considered as a centralised repository...

November 20, 2019.

Why switch to a more modern ci/cd system?

I’ve recently been working a lot with Gitlab, and though it would be useful to explain the pros and cons offered a traditional ci/cd system, such as Jenkins against one...

November 9, 2019.

Reflected Cross Site Scripting

Cross Site Scripting (XSS) attacks are a type of injection attack, where scripts are injected into a tusted website application. There are sub categories of XSS attack, which work very...

November 7, 2019.

Using AWS App Mesh

AWS App Mesh is a managed version of Service Mesh that AWS provide. App Mesh is designed to be used purely for traffic management of services running in AWS. What...

November 7, 2019.

How to hot reload grapphql using webpacks Hot Module Replacement Plugin

Hot Module Replacement will exchange, add or remove a part of your application, such as modules or in our case graphql schema and inner workings, without the need of a...

October 26, 2019.

Hack Manchester 2019 - Runner up for best companies

This year I decided to enter hack manchester with some of my old work friends from Zuto (this year is my fourth time running!). The team got to MOSI at...

September 11, 2019.

Autoscaling Fargate tasks, outside of a load balancer

I recently worked on a platform that required scaling. In the end, we chose to look at alternative options for autoscaling, but this was the approach we attempted. It would...

September 11, 2019.

Using boto to consume a service assuming a role within another account with terraform.

I came across a scenario today, when working with a task management platform, where the task runner (which was using python) needed to be able to assume a role, in...

August 20, 2019.

setting up airflow using celery executors in docker

I’ve recently been tasked with setting up a proof of concept of Apache Airflow. What is apache airflow? Apache Airflow is an open-source tool for orchestrating complex computational workflows and...

August 15, 2019.

SSH Tunneling

Today i had to connect mysql workbench to a mysql instance hosted within RDS. The rds instance didn’t allow public access, and the only way I could connect, would be...

August 6, 2019.

Looking for a good reverse proxy solution

I have recently been working on a project to find which reverse proxy is best to sit in front of many services. The idea being that within the proxy, we...

July 10, 2019.

Automation of deployments and blue green

What are blue green deployments? Traditionally, a blue-green deployment was described as a technique that reduces downtime and risk by running two identical production environments called Blue and Green. At...

May 24, 2019.

Capture the flag javascript challenges 1 to 4

I’ve been playing more of the challenges on the ringzer0 website, but this time looking at the javascript challenges. Here is how I solved the first 4 challenges. https://ringzer0ctf.com/challenges Challenge...

May 18, 2019.

Playing capture the flag - coding challenges - sql injection 1

I recently have been playing some capture the flag games, I found a site called ringzer0 who host many different types of coding challenges. It mentions on their site: RingZer0...

May 17, 2019.

The pros and cons of using graphql instead of rest

Pros Data Size - You can query for only the data you require, and you will be returned exactly that. This could be done in rest also, but it’s not...

May 14, 2019.

Alerting on a failed fargate task - built using cloud formation

Most of my posts on my blog so far, with regards to infrastructure as code have been related to terraform, so I thought I would write a post about cloud...

May 9, 2019.

Understanding HMAC authentication and applying it to HTTP

I have been reading up on hash-based message authentication code (HMAC), and how amazon use it to secure their apis. Why HMAC? The idea behind HMAC is to generate a...

April 2, 2019.

Returning large data iteratively in dotnet core.

Scenario We have an export function, which pulls data from elasticsearch, and uses graphql to load relational data into the data. It returns a very large dataset. The problem with...

March 30, 2019.

Blue Green deployments, utilising Terraform and AWS ECS

What is a blue green deployment? Blue-green deployment is a technique that reduces downtime and risk by running two identical production environments called Blue and Green. At any time, only...

March 29, 2019.

What is it like to work remotely?

What is is like to work remotely? I had visions of working in the garden on sunny days, a different coffee shop every day, working late some days and finishing...

March 24, 2019.

Living in Utopia with Mainline Development

I love a good merge conflict me …said no developer ever!! Yet its something that happens all the time. There are some great options out there, but the last 3...

March 15, 2019.

dotnet core code coverage using AltCover and ReportGenerator

AltCover is awesome!! Code coverage for dotnet core Some of the traditional code coverage tools for .Net have still not yet been ported for dotnetcore. A while back, I was...

March 15, 2019.

Analysing network traffic for your ALB, using elasticsearch and lambda

At my job, I utilised my 10% hack time at work to build a system to analyse network traffic. I know there are tools out there like beats, which can...

March 4, 2019.

Passing keys to dockerbuild from build agent

At ditto, we use a docker build agent, for building our docker images and pushing to our private docker repository. This works really well for us, but there are times...

February 4, 2019.

#GraphQL and why its awesome

TLDR - GraphQL is awesome I was quite late to the party with GraphQL, not really investing much learning time into it until recently. It most definitely has its use...

February 3, 2019.

#DevOps and the evolution from Developer to multi skilled Engineer

TLDR - DevOps is awesome, DevOps is not scary, and using the right tools and patterns will allow you to easily transition from Software Developer to Software Engineer and keep...

January 17, 2019.

ECS AWS WebApps - A simple pattern for deploying anything quickly using terraform

One of the best things I like about infrastructure as code, and terraform in particular, is the ability to quickly, and securely deploying something, based on a pattern that you...

December 12, 2018.

Plugging an existing authenticated session into GraphQl

I’ve recently started working with GraphQL, and I really like it! In work capacity, we have so far used it to talk to a few different services and databases. One...

November 21, 2018.

Using dotnet to read gzip streams from s3

TLDR - Theres a bug in the dotnet gzip compression library, and it doesn’t seem like it’s going to be fixed anytime soon So I came across a bug recently...

November 13, 2018.

Hack Manchester 2018 - Runner up for GCHQ challenge

Me and a couple of guys from the office entered HackManchester (me for the third year running). We also gained an additional member of the team, Jon was looking to...

October 26, 2018.

Processing an SQS queue without manual polling, using lambda with event source triggers, built using terraform

I was recently working on something that would process a series of messages from a queue. In the past, this has involved some kind of polling, such as a cron...

October 3, 2018.

Tutorial of using terraform import for syncing terraform state

It’s worth noting that its always much simpler to create the terraform first, you would only ever import when you need to do something reactive, like an emergency release There...

September 14, 2018.

Setting up an alb as an alternative to nginx, using terraform

Setting up NGINX or Apache can be a pain. I’ve been working with AWS Application Load Balancer recently, and setting up routing is fairly straightforward, and in my opinion, if...

August 7, 2018.

Setting up CI environment using Docker, ECS and terraform for Thoughtworks GoCD

One of my first tasks for my new job at Ditto Music, was to setup the CI server in a way that would be easier to manage and scalable. There...

July 11, 2018.

Elasticsearch Bulk Api for data migration

At work, I recently worked on a project to move from a self hosted elasticsearch cluster in Azure, to a managed elasticsearch cluster in AWS. The cluster contained analytics tracking...

July 5, 2018.

Setting up a container cluster in aws with services using terraform

I recently setup an ecs cluster in AWS, which allowed me to run containers on the cluster, rather than having to run individual ec2 instances per application. Create a Dockerfile,...

June 21, 2018.

Performance Testing an Api using Artillery and Faker

I needed to do some performance testing on a lambda, behind an api gateway to just prove that it scaled well, and my initial thought was to turn to something...

June 5, 2018.

Blockchain and cryptoeconomics

Blockchains have recently gained popularity in several industries, and although I am yet to use them in my day to day, it’s something I would like to in the short...

May 31, 2018.

Acceptance Test Driven Development

This post I’ve meant to have wrote about for years now, but never got round to it. I came across a slide deck I created when I first started at...

May 30, 2018.

Terraform dependency graphing visualisations

There a really cool feature of terraform that I only just found out about, Terraform Graph. Terraform Graph outputs a visual dependency graph of the resources that are inside it’s...

May 30, 2018.

Setting up a serverless api using Api Gateway, Aws Lambda, DotNetCore and terraform.

What I’m aiming to do, is create a gateway api, that will call multiple lambdas. It will look similar to a standard web api, except it will be all serverless....

May 22, 2018.

AWS security and IAM Policies

AWS take security very seriously, and there are many physical security measures that are taken place to improve security, such as the location of the AWS data centres, are not...

May 14, 2018.

Library for testing an AWS lambda locally with C#

Recently I’ve been working on a library that will allow you to run an AWS lambda locally, as you would within an API Gateway. This is really useful for testing...

May 10, 2018.

Load Balancers in AWS

There are two PAAS load balancer offerings within AWS. The classic load balancer and the application load balancer. In the past, the classic load balancer was called the elastic load...

May 9, 2018.

Setting up the elbk stack

It’s worth noting, this post refers to ELBK stach version 5, the latest can be found here: https://www.elastic.co/. Also there are PAAS offerings, which means you can have a cloud...

May 7, 2018.

Setting up a simple cloud in AWS

Setting up a simple cloud I do a lot of terraform configuration at work, but never had a chance to look at how to provision stuff within the AWS management...

March 11, 2018.

Webpack

What is Webpack? According to the Webpack documentation: Webpack is a module bundler. It takes modules with dependencies and generates static assets representing those modules. It focuses on Code Splitting...

March 10, 2018.

Module Loaders and Transpilers

This post hopes to explore the past, current and future state of build processes in front end technologies. When working with node, its commonplace to use modules, but what I...

March 4, 2018.

History of webpage layouts

1991 - The first website Back in August 1991, the first website was a simple text-based page with a few links. The first ever webpage served only to tell people...

February 28, 2018.

Snow Day

Its been a very cold an snowy day, had the day off from work, so making the best of the time playing in the snow with my daughter!

February 26, 2018.

Flexbox

I found myself doing some front end work, after months of backend and migration work. I felt a bit rusty when it came to layouts, and although using a prebuilt...

February 2, 2018.

Why use jekyll as a blogging platform?

I’ve finally got round to migrating my blog over to Jekyll, I must say I feel like its much better than blogger, (I can’t believe I used it for so...

January 20, 2018.

Front End North Conference 2018

I went for a day trip to sheffield to attend the FrontEndNorthConference In summary, it looks like we are already doing what the rest of the industry suggests as best...

October 20, 2017.

HacManchester 2017

GCHQ Winners, Hack Street’s Back! We entered the competition in 2016 and close very close to winning an award, but this year we won!! The team was made up of...

October 1, 2017.

Log Rotate in Unix

Recently I’ve been doing some work with analytics and tracking and came across log rotation in Linux. The problem I came across was when parsing nginx logs to build up...

August 20, 2017.

Building a docker container - nginx

Recently I was working with docker, and I found it a bit painful. I think I just jumped straight in without any real knowledge, but after some time to reflect...

February 10, 2017.

App Insights Analytics

I’ve been playing around with azure app insights analytics again. I focused on seeing if there is a way that i could work out a users journey through a website,...

January 25, 2017.

Mob Programming

Last night I attended a session on Mob Programming and Working well together, with guest speaker Woody Zuill, who claims to be one of the founders of the practice. The...

November 3, 2016.

Hack Manchester 2016

I entered the hack mancester competition with a few other people from Zuto, Tom McGiveron, Nathan Smith and Tom Pennington. It took place on the weekend of 29th-30th October and...

January 12, 2016.

Powershell Iterators and Selectors

I’ve used powershell quite a lot, I find it most useful at work when setting up build pipelines for team city, its really handy when creating build steps and you...

January 11, 2016.

FizzBuzz kata in FSharp

https://github.com/craiggoddenpayne/FizzBuzz-FSharp It’s been a while since I got to use a functional language, I did a lot of training on scala, but never really used F#, so I gave it...

January 4, 2016.

Specflow Run Transforms

So I came across a problem today with SpecRun and concurrency issues. It started after I wrote a suite of end to end smoke tests, that are to be run...

January 3, 2016.

Upgrading to OxygenOS

So its been a while since I updated my phone firmware, for some reason on the OnePlusOne if you have a custom firmware, you still get prompted for an update,...

December 27, 2015.

Javascript call and apply

Call and apply, it’s all about “this” and setting the context when inside a function. It’s important to know how and why you would need to do this, especially if...

December 15, 2015.

Stacked 2015 conference

At the time I never got to write up about the Stacked conference I attended, so here goes: I was looking at the various conferences going on this year, and...

October 12, 2015.

Mobbing FizzBuzz

I was having a chat with another developer at work about mob programming (something that’s a bit taboo where we work). I ran a developer huddle for our team and...

December 15, 2014.

12 days of xmas kata

I found a nice seasonal challenge for the end of the year - printing out the lyrics to the ‘12 Days of Christmas’: On the twelfth day of Christmas, My...

September 6, 2013.

JINQ

I was a bit bored so started writing my own linq style extensions for javascript. Its pretty cool so far, they work nicely! Github More to follow!!

August 17, 2013.

Node Rest Template

I’ve been working on a project so that I can get started quickly when wanting to write a quick rest service. I used node, since its my new favourite thing,...

May 27, 2013.

Colourstrap/Colorstrap

My new project! colourstrap https://github.com/craiggodden-payne/colorstrap Eventually i will fork between the two, its just annoying that the american spelling and british spelling of colour have to differ!! I’ve started working...

March 15, 2013.

Jade server for node js

I have been playing around with Node.js again, and have create a Jade web server. Jade is a html templating engine similar to haml, that allow you to write html...

March 6, 2013.

Blackout

I got sick of building snippets of javascript to block out sections of webpages when ajax calls are made etc. I built a jQuery plugin to do this for me,...

March 2, 2013.

Google Maps Api

I’ve been playing about with the Google maps API. Here is an example - I have set the latitude and longitude to show all the bars in the northern quarter...

January 18, 2013.

JQuery plugin PageScraper

I’ve been reading a lot lately about JSONP and cross domain scripting and the massive pain it causes when doing things like page scraping. I created my own PageScraper JQuery...

October 1, 2012.

Functional programming in scala

Its been a while since i managed to get the time post on my blog, I’ve been really busy with work and home life, I’ve recently enrolled in an online...

September 30, 2012.

Javascript Convention

I’ve been writing my js13k submission whenever i have had chance over the last few days, and it’s made me think more about the way i write javascript. I haven’t...

September 10, 2012.

JS13k coding competition | Agent XIII

This should be fun… there’s a competition running at the moment (ending 13th Sept) to create a full javascript / Html5 game that is under 13k in size (minified) Js13kGames...

April 15, 2012.

Woo! RaspberryPi!

So i finally got my pi! I went to micro direct at the weekend, turns out they have lots in stock!! I cancelled my order with RS components as it...

August 13, 2011.

Online Reputation

So I’ve been reading a lot recently about “Online Reputation” and how important it is going to be in the future, Apparently its common now for employers to do a...

January 1, 2000.

You are being redirected...

You are being redirected