Tuesday, 3 February 2015

Keeping a windows process after a jenkins job

One of our jobs on Jenkins is to deploy and startup an application server on a remote slave. However we were having 2 problems with it:
1. the job wasn’t finishing
2. when we terminated it from jenkins, it killed the process it had spawned (the application server).

I spent ages butting my head against Jython on windows, which in hindsight, I could have saved a lot of time if I'd seen how little of the functionality I need from python/windows was implemented in jython/windows.

(1) was solved by getting the start command to write stdout/stderr to a file (irrespective of whether anything was written).

(2) I tried many elaborate solutions, still thinking it was a jython or windows related problem. After seeing it mentioned in blog posts a few times I finally got what they were saying.
Jenkins has a section on their site: Spawning processes from build. Now the key thing here I missed was the BUILD_ID environment variable. So unset it and as long as you've spawned a new process it should fine.
The simple solution should be start_background.bat
set BUILD_ID=
start /B "" cmd /C %*
And just pass your full command to start_background.bat

Unfortunately that didn't work in my odd case, so here's my overkill solution for those that need it:
# Use csscript.exe to trigger the call async and not attached to this process as os.spawnl(command, os.P_DETACH)
    def windowsAsync(self, systemCommand, jobName):
        cmd = open(jobName + '.cmd', 'w')
        cmd.write("set BUILD_ID=\n")
        cmd.write(systemCommand + "\n")
        cmd.write("exit\n")
        cmd.close()
        startCommand = 'start "" /B cmd /C ' + jobName + '.cmd'
        # saving the output to a temporary file to make sure we don't have handles. don't know if this acutally does it
        tOut = tempfile.NamedTemporaryFile()
        tIn = tempfile.NamedTemporaryFile()
        print "Using output temp file: " + tOut.name
        print "Invoking OS command: " + systemCommand
        print "Embeded command: " + startCommand
        process = subprocess.Popen(startCommand, shell=True, stdin=tIn, stdout=tOut, stderr=tOut)
        #process.wait()
        print "Sleeping"
        time.sleep(30)
        
        print "--- consoleOutput start:"
        # Rewind and read the text written
        # to the temporary file
        tOut.seek(0)
        lines = tOut.read()
        print lines
        tOut.close()
        sys.stdout.flush()
        tIn.close()
        
        lines = ""

No comments:

Post a Comment