DevGrok

Running Jenkins using ECS tasks to run worker nodes has been documented before , however there aren’t any up to date examples, nor  provide separation of the master and salves. This post is fairly up to date deployment using the newer deployment techniques offered by AWS. Even if your not using Jenkins, being able to create CloudFormation templates using the new EC2 launch options is helpful if you’re using many spot instances, as you will most like be experiencing instance type availability fluctuations. Key features of this deployment Both worker nodes and master node run on ECS: the master as a service and slaves as dynamically added tasks The master node runs on it’s own dedicated cluster; it's file system store is only mounted on and accessible by the master The job can launch and run build docker images from within an already running container The worker nodes can also spawn build agents (docker containers) using the "new" Jenkins pipeline syntax; where the ...

In our team, we can't run continuous deployment into our testing environments. Like most enterprises with large/legacy back-end systems, we only have a few up-stream instances running with populated data. That results in having a finite set of test environments: qa-1, qa-2, sit-1, sit-2, etc. QA would usually be stubbed out so having one per feature branch wouldn't be a problem but for SIT, having a consistent & known environment for integration testing makes this difficult. Previously, I used Bamboo and it had the concept of Releases, Environments and dedicated Deploy Jobs. These allowed for builds to be made into releases (manually or automatically) and have those release deployed to specific environments using predefined deploy jobs, with only one release being recorded as deployed to an environment at a time. The advantage of this was it allowed our testers to easily see what was currently deployed and where, without having to dig through every job. Jenkins 2 was re...

One of our jobs on Jenkins is to deploy and startup an application server on a remote slave. However we were having 2 problems with it: 1. the job wasn’t finishing 2. when we terminated it from jenkins, it killed the process it had spawned (the application server). I spent ages butting my head against Jython on windows, which in hindsight, I could have saved a lot of time if I'd seen how little of the functionality I need from python/windows was implemented in jython/windows. (1) was solved by getting the start command to write stdout/stderr to a file (irrespective of whether anything was written). (2) I tried many elaborate solutions, still thinking it was a jython or windows related problem. After seeing it mentioned in blog posts a few times I finally got what they were saying. Jenkins has a section on their site:  Spawning processes from build . Now the key thing here I missed was the  BUILD_ID environment variable. So unset it and as long as you've spawned a new...