Port Forwarding on Mac OSX

If your running vagrant and you're forwarding traffic to vagrant over 8080, but you really prefer to hit port 80, you can use Mac's pfctl function. 

Here's a couple of links that you might find helpful
https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man8/pfctl.8.html
http://krypted.com/mac-os-x/a-cheat-sheet-for-using-pf-in-os-x-lion-and-up/

I was reading this article http://salvatore.garbesi.com/vagrant-port-forwarding-on-mac/ and it suggested adding a vagrant plugin, but it's a ruby gem. Hard to imagine, but the gem failed to install.

You can still implement port forwarding. Create a pfctl.conf file in your vagrant folder:

echo "rdr pass on lo0 inet proto tcp from any to 127.0.0.1 port 80 -> 127.0.0.1 port 8080
rdr pass on lo0 inet proto tcp from any to 127.0.0.1 port 443 -> 127.0.0.1 port 8443" > pfctl.conf

To run this, you first need to enable pfctl

pfctl -e -f pfctl.conf

Once you enable the firewall rules, you can hit your vagrant box by going against localhost. 

To disable pfctl: 

pfctl -d

Searching Log files with AWK

I was digging through my notes and came across this little nugget. Say you have a log file and that log file entry has URL encoded values that you would prefer to see decoded. Here's the AWK I used to URL decode:

awk -F ^C '$4 ~ /SearchingFor/ {print $4}' access_log | awk '
{
    str = $0
    while (match(str,/%/)) {
      L = substr(str,1,RSTART-1) # chars to left of "%"
      M = substr(str,RSTART+1,2) # 2 chars to right of "%"
      R = substr(str,RSTART+3)   # chars to right of "%xx"
      str = sprintf("%s%c%s",L,hex2dec(M),R)
    }
    printf("%s\n",str)
    
}
function hex2dec(s,  num) {
    num = index("0123456789ABCDEF",toupper(substr(s,length(s)))) - 1
    sub(/.$/,"",s)
    return num + (length(s) ? 16*hex2dec(s) : 0)
}'

I found this on the web and I post it here so I don't loose it. I would love to give credit if I could remember where I got it.

How to merge in specific files from one git branch to another

I found this write up very helpful:

http://jasonrudolph.com/blog/2009/02/25/git-tip-how-to-merge-specific-files-from-another-branch/

git checkout <branch> <file>

So, say you have a feature branch foo and your master branch and you want to get one file, src/bar.c into master

git checkout master
git checkout foo src/bar.c
git commit -m "merging bar.c from foo branch" src/bar.c

That's it. 

Git: How to clone a repository using a token on the command line

I was creating a vagrant development instance and I wanted clone from our private repository and this was something I intended to share. So, I didn't want anyone to have to log in and I didn't want to have manual steps. Since we already had a token for our group, I wanted to be able to use the token to clone and do git pulls on the development instance for all of our puppet configs. I found a solution on the web that led to the answer.


Create your token: Read up on how to create a token and create your token on github - it's easy. I used this link: https://help.github.com/articles/creating-an-access-token-for-command-line-use/

Then you can use your token to do anything it allows you. For example:

git clone --recursive https://TOKEN-GOES-HERE:x-oauth-basic@github.com/private-repo/puppet-configs.git /etc/puppet

The site that helped me was this one: 

http://rzrsharp.net/2013/07/02/private-github-repos-with-npm-and-heroku.html

Git: How to delete a tag off the remote repo

If you have a tag named '12345' then you would just do this:

git tag -d 12345
git push origin :refs/tags/12345

That will remove '12345' from the remote repository.

Found that wonderful piece of help here:

http://nathanhoad.net/how-to-delete-a-remote-git-tag

Calling a jenkins job with a bash script

I don't care to use a mouse when I'm working on a project unless I have to. As nice as Jenkins is, if you have to re-run a job often, clicking through their UI can get old fast. I had trouble getting the API access working with a script, so I better write down what I did.

First things, you need to enable script triggers at the job level. Configure the job and under Build Triggers, select "Trigger builds remotely" and add an API key. See my earlier post for one or two options to generate a job token.

Next thing you need to do is set/get your personal API token from Jenkins. You could create another account if you choose to, but you will need an account that has access to execute the action you want to do e.g. have a script to run a job. Click on your login name and then configure > API Token > Show API Token. 

I took this from one of the jenkins posts but I don't remember which one. I use curl instead of wget and handle parameter jobs or parameterless jobs. There's more you could do to clean this up, but it works.

	#!/bin/bash ########################################################################
	# This is an example how to call a jenkins job using bash
	#
	# PreReqs:
	# 1: Add a token to your jenkins job. Configure job,
	# "Build Triggers - Trigger builds remotely (e.g. from scripts)"
	# One way to generate a job token
	#
	# $> r="";for e in $(seq 1 5); do r="${r}$(od -vAn -N4 -tu4 < /dev/urandom|sed 's| ||g')"; done;echo $r|openssl sha1 -sha256
	#
	# 2: Go get your own personal token
	# You can click on your name, 'Configure > API Token > Show API Token
	#
	# 3: Run your job e.g. run one that has parameters
	# ./run-jenkins-job.sh login <persoal-token> jenkins-job-name <job-token> 'PARAMA=value&PARAMB=value'
	#
	# run one without params
	# ./run-jenkins-job.sh <login> <login-token> <jenkins-job-name> <job-token>
	#
	# Add any job parameters there.
	########################################################################
	USER="$1"
	TOKEN="$2"
	JOB="$3"
	JTOKEN="$4"
	PARAMS="$5"
	function usage {
	echo "$0 <jenkins-login> <jenkins-login-token> <jenkins-job> <job-token> 'job_param_a=value&job_param_b=value'"
	}
	if [ "" == "${USER}" ]; then
	usage
	exit 1
	fi
	if [ "" == "${TOKEN}" ]; then
	usage
	exit 1
	fi
	if [ "" == "${JOB}" ]; then
	usage
	exit 1
	fi
	if [ "" == "${JTOKEN}" ]; then
	usage
	exit 1
	fi
	# jobs that have params need a different url and the params need to be on the query string
	if [ "" == "${PARAMS}" ]; then
	NOTIFY_URL="job/${JOB}/build"
	else
	NOTIFY_URL="job/${JOB}/buildWithParameters?${PARAMS}"
	fi
	CRUMB_ISSUER_URL='crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)'
	function notifyCI {
	CISERVER=$1
	# Check if "[X] Prevent Cross Site Request Forgery exploits" is activated
	# so we can present a valid crumb or a proper header
	HEADER="Content-Type:text/plain;charset=UTF-8"
	CRUMB=$(curl --user ${USER}:${TOKEN} ${CISERVER}/${CRUMB_ISSUER_URL} 2>/dev/null)
	if [ "$CRUMB" != "" ]; then
	HEADER=$CRUMB
	fi
	curl -X POST ${CISERVER}/${NOTIFY_URL} --header "${HEADER}" --data-urlencode "token=${JTOKEN}" --user "${USER}:${TOKEN}"
	echo "Done"
	}
	# The code above was placed in a function so you can easily notify multiple Jenkins/Hudson servers:
	notifyCI "http://jenkins-server"

Generating a token

I found a couple of ways to quickly generate a token for use with APIs. I needed a unique token for my Jenkins jobs and found these two useful.

r="";for e in $(seq 1 5); do r="${r}$(od -vAn -N4 -tu4 < /dev/urandom|sed 's| ||g')"; done;echo $r|openssl sha1 -sha256

d81697b6322a81a7fb19e0ef1141f534da0634244e76b5590332a1a186c7c4a9

r="";for e in $(seq 1 10); do r="$r${RANDOM}"; done; echo $r|openssl sha1 -sha256