Isthmus – connecting REST APIs

/ˈɪsθməs,ˈɪstməs,ˈɪsməs/
  1. a narrow strip of land with sea on either side, forming a link between two larger areas of land.
  2. a narrow organ, passage, or piece of tissue connecting two larger parts.

Isthmus connects two REST APIs, transforming the incoming payload to the outgoing one using a FreeMarker template.

Are you using git, Jenkins and maybe Atlassian JIRA ®? Are you communicating via Slack? Chances are you've found some useful plugins, but for a few use cases there just isn't anything suitable out there. Isthmus covers the gap between the tools we use for development on a daily basis. It allows to automate those "if this then that" rules your team has, but occasionally fails to observe. Isthmus takes an HTTP JSON payload, either triggered via webhook or scheduled event, and transforms it into another HTTP JSON payload.

Use Isthmus if:

I'm hooked .. download

Recipes

Periodically query JIRA, and report results to Slack

Assumptions
  • JIRA is hosted at https://www.mycompany.com/jira
  • You'll use a JIRA filter named 'Filter for Isthmus'
  • You want an update weekly, on Thursday evening (7.15pm)
What this shows
  • Isthmus can run HTTP GET per CRON schedule, and process the response to another REST API
  • The REST API payload is completely flexible thanks to using a FreeMarker template
Example setup
  1. Set up an incoming webhook on Slack
  2. Define a filter in JIRA (you could also query directly using JQL, but using a filter allows for adapting the query without changing your Isthmus setup)
  3. Define a new scheduled rule with:
    • CRON expression: 0 15 19 ? * THU
    • Query URL:
      https://www.mycompany.com/jira/rest/api/2/search?jql=filter='Filter for Isthmus'
    • Provide username and password with access to JIRA and the provided filter
    • FreeMarker template
      {
                "text": "Tasks in progress for tomorrow:\n<#list issues as issue><https://www.mycompany.com/jira/${issue.key}|${issue.key} - ${issue.fields.summary}>\n</#list>",
                "username" : "Isthmus",
                "icon_emoji" : ":heavy_exclamation_mark:", 
                "channel" : "jira-updates"
            }
    • Method: POST
    • https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX

Update JIRA when Jenkins finishes

Assumptions
  • Jenkins is hosted on the same server as Isthmus
  • JIRA is hosted on the same server as Isthmus, at http://localhost:8081/jira
  • Our Jenkins job deploys an artifact, so we'll set a JIRA label 'deployed'
What this shows
  • API URLs are also treated as FreeMarker templates and are thus flexible
This one is a bit tricky because there are several ways how a Jenkins notification can be set up.
  • The Notification plugin sends JSON via HTTP, but in our tests it never reported any SCM changes
  • The precursor to above plugin, jenkins-notify-plugin, allows for templating the payload, but requires manual installation (and seems to be discontinued)
  • Create a JSON payload from scratch and use cURL .. ok, let's do that
Example setup
  1. Extend your declarative pipeline with
    • Implement some form to retrieve your SCM change list as eg. changelistJSON
    • script {
              payload = changelistJSON()
              echo "Sending JSON to Isthmus: ${payload}"
              sh "curl -d '${payload}' -H "Content-Type: application/json" -X POST http://localhost:8087/isthmus/webhooks/jenkins"
            }
      (the -X POST here is unnecessary, but illustrates that we're sending POST)
  2. Define a new webhook rule with:
    • /webhooks/jenkins
    • 'Payload must match' remains empty
    • 'Path to payload root for template' stays empty (indicating we're using the full payload)
    • FreeMarker template
      {
              "update": {
                "labels": [{
                  "add": "deployed"
                }]
              }
            }
    • Method: PUT
    • http://localhost:8081/jira/rest/api/2/issue/${key}
    • Username + password to access JIRA

Update JIRA on git push

Assumptions
  • Common practice is to reference the (one) JIRA task in the commit message using the task key
  • Success of failure of Isthmus update must not prevent a git push, so we'll use a 'post-receive' hook
  • On pushing a change, we want to remove any 'deployed' label present, and set a 'code-change' label
What this shows
  • Incoming payloads can be filtered using RegExp (or JSON Pointer for JSON payloads). The matched groups are available as group1, group2, ...
  • The payloads don't have to be JSON, text/plain also works
Example setup
  1. Define a 'post-receive' git hook:
    #!/bin/bash
          read oldrev newrev _branch
          branch=$(echo $_branch | sed 's/.*\/\([a-z0-9][a-z0-9]*\)$/\1/')
          commit_message=$(git log -1 --pretty=format:'%h %cn: %s%b' $newrev)
          curl --header 'Content-Type: application/json' --data "$commit_message" http://localhost:8087/devtaskhub/webhooks/git &
  2. Define a new webhook rule with:
    • /webhooks/jenkins
    • Set a 'Payload must match' to .*FUN-.* with type RegExp
    • Set 'Path to payload root for template' to .*?: ([\w-]+).*
    • FreeMarker template
      {
              "update": {
                "labels": [{
                  "add": "code-changes"
                }, {
                  "remove": "deployed"
                }]
              }
            }
    • Method: PUT
    • http://localhost:8081/jira/rest/api/2/issue/${group1}
    • Username + password to access JIRA

Setup, tips and tricks

A typical command line (startup.sh) might be
java -Disthmus.logincoming=true -jar isthmus-0.0.91.jar --server.port=8087 --server.servlet.context-path=/isthmus &
For everyday operations, you'll want to setup something in init.d, or cron @reboot, or whatever approach you prefer.
Jenkins
Jenkins setups can be a bit tedious, so we'd like to point out to invaluable tools where we'd rather bore you with something obvious than fail to mention them.
  • Jenkins has a built-in help for pipeline syntax. If your Jenkins job is hosted at http://localhost/jenkins/my-pipeline-job, then the pipeline syntax is explained at http://localhost/jenkins/my-pipeline-job/pipeline-syntax/
  • When editing a Jenkinsfile, make use of the Replay function (introduced with Pipeline 1.14). In the above setup, that would be at http://localhost/jenkins/my-pipeline-job/999/replay/
Questions?
Please let me know if you have questions or feature requests. You'll find contact information at want.ch