Posts

Resizing root partition on CentOS 6 in the cloud

If your unfortunate enough to still be using CentOS 6 in AWS than more than likely be stumped why the root partition isn’t resizing like most other distros you launch.

Installing the new Cloudwatch Agent for ECS

If you want to gather log files on your ECS hosts running a Amazon ECS-optimized AMI, then there’s instructions on how to install the Previous CloudWatch Logs Agent using cloud-init and some user data. The old Logs Agent is still supported but the new Unified CloudWatch Agent is recommended as it is touted to be faster but more importantly, allows easy collection of instance metrics. To install it, simply launch an EC2 instance using one of the Amazon ECS AMI’s, and put this in the user-data section (expand the advanced configuration of the launch wizard). Once the EC2 instance is started up, just simply create an AMI from it, shutdown the EC2 instance and use this new AMI for you ECS clusters or for your Batch Compute Environments. With a bit of modification, this script could be used to install and configure it on the launch of each ECS instance, but having a pre-baked AMI is a bit cleaner.
This script also increase the default docker container size to 100GB, using the ECS Agent st…

Jenkins build and multi-environment deploys

Image
In our team, we can't run continuous deployment into our testing environments. Like most enterprises with large/legacy back-end systems, we only have a few up-stream instances running with populated data. That results in having a finite set of test environments: qa-1, qa-2, sit-1, sit-2, etc. QA would usually be stubbed out so having one per feature branch wouldn't be a problem but for SIT, having a consistent & known environment for integration testing makes this difficult.

Previously, I used Bamboo and it had the concept of Releases, Environments and dedicated Deploy Jobs. These allowed for builds to be made into releases (manually or automatically) and have those release deployed to specific environments using predefined deploy jobs, with only one release being recorded as deployed to an environment at a time. The advantage of this was it allowed our testers to easily see what was currently deployed and where, without having to dig through every job.

Jenkins 2 was rel…

Using web component polyfills with template tags

I've been playing around with using <template> tags and how well they work with the current Web Component (Custom Elements) polyfills.

My main motivation for going for Web Components instead of something like React or Angular is that I'm currently developing a chrome extension. I wanted the code base to be as small so that it didn't slow down devtools and increase the frequency of hands. Plus I think it's going to be the natural progression from the current React/Angular/etc components - especially with HTTP 2.0's server push of dependant files removing the need for tools like webpack by allow all dependant files to be automatically sent in response to one request.

I immediately hit problems using custom elements in a chrome extension as they're disabled by default. So in order to use them I had to forcefully polyfill the existing API, it took a bit of fiddling  but now works with both libraries I looked at.

Next, using template tags an import link html f…

How to chain an ES6 Promise

Node.js uses async functions extensively, as it based around non-blocking I/O. Each function takes a callback function parameter, which can result in some messy, deeply nested callback functions if you have to call a bunch of async functions in sequence. Promises make these callbacks a lot cleaner.

ES6 (or ES2016) Promises are a great way of chaining together asynchronous functions so that they read like a series of synchronous statements.

There's already some great posts on Promises, 2ality has a good intro to async then detail of the api, so I won't rehash that article. However, after starting to use them for cases more complicated than most examples, it easy to make a few mistaken assumptions or make things difficult for yourself.

So here is a more complicated example showing a pattern I like to follow. In this example, I'll use the new Javascript Fetch API, which is an API that returns a Promise, allowing you to make async HTTP calls without having to muck around with …

Adding MDC headers to every Spring MVC request

Mapped Diagnostic Context (MDC) logging allows you to set attributes associated with current thread, so that SLF4J (via your logger implementation library) can log those attributes with each logging statement without it having to be specified.

For example you could configure logback to log the sessionId on every statement, which is really handy when using a log indexer such as Splunk. This would allow you to easily see all the requests made by a user, for a given session. To use with logback, you'd set the pattern to %-4r [%thread] %-5level sessionId=%X{sessionId} - %msg%n

Setting these attributes for each entry point would be a pain so one way would be to implement a ServletRequestListener, which would allow setting the attributes at the start of the request and removing them again at the end of the request. (Note: It's important to remove the attributes afterwards, as threads are re-used by application servers and will give misleading logging statements)
If you're us…

Populating stored procs into a HSQL DB

I recently encountered a problem trying to load stored procedures into a HSQL DB used for testing. The problem was caused by the script runner provided by spring which separates each statement to be executed in a script file by a semicolon. If a stored proc has statements inside it (which most do), then the proc isn't executed as a single statement. This is further compounded by each statement executed must be understandable by JDBC. For example the following stored proc causes issues: CREATE PROCEDURE MY_PROC(IN param1 VARCHAR(30), OUT out_param VARCHAR(100)) READS SQL DATA BEGIN ATOMIC SELECT the_value INTO out_param FROM my_table WHERE field = param1; END .; This problem is solved by using the script runners provided by HSQL in the "org.hsqldb:sqltool" dependency as they parse can correctly parse the scripts containing stored procedures. Here is a Spring Boot test, using an in memory database but using HSQL's script runners: @RunWith(SpringRunner.class…