Tuesday, April 12, 2016

Getting Started with RPMs

Red Hat Package Manager (RPM) is a way to package and deploy software on Red Hat variants that’s similar to Advanced Packaging Tool (APT) on Ubuntu variants. To create RPM files, you can use third party tools like Ant or FPM, but with a little bit of effort you can quickly write your own spec files. This paper isn’t intended to be a complete reference on RPM files. There are a couple of great resources out there to get the full scoop on RPMs and I’ve added them at the end of this paper.
To make an RPM you need two things, a spec file and the rpmbuild program. You can get rpm-build easily enough e.g.



$> yum install –y rpm-build rpmdevtools
$> rpmdev-setuptree


The last command will create your rpm build tree e.g.

/home/user/rpmbuild
/home/user/rpmbuild/RPMS
/home/user/rpmbuild/SPECS
/home/user/rpmbuild/BUILD
/home/user/rpmbuild/SOURCES
/home/user/rpmbuild/SRPMS


If you put your tree somewhere else or if you want to be explicit, add an .rpmmacros file to your home directory.

$> echo “%packager First Last 
%vendor Your Company
%_topdir ${HOME}/rpmbuild” > ~/.rpmmacros

Now you have the basic requirements and we can start writing spec files. However, before we dive into the spec file it helps to understand that spec files have macros and that macros can be a single value or a multi-line value. Here is how we define a variable in a spec file:

%define name value

You can conditionally set the variable like so:

%{!?_version: %define _version 1.0.0}


When you use the variable you use you use the %{} syntax e.g.

Version: %{_version}


You can conditionally use a variable e.g.

Release: 1%{?dist}


The %{?dist} syntax means, output the value of “dist” if it’s set, otherwise output nothing. rpmbuild will fail if you reference macros that are not defined. If you want to see what the current value is of a predefined variable you can run the following command:

$> rpm --eval '%{_rpmdir}'


To get a list of all pre-defined variables, see the links at the end. If you want to see everything you can use either of these two commands:

$> rpm --showrc | less
$> rpmbuild --showrc | less


I piped them to less in my example because there’s a lot of information when you run either of those commands. Since we can define variables and show them, if you wanted to play around you can:

$> rpm --define '_some value' --eval '%{_some}'
value


You can also include variables listed in a separate file

%include some.spec.file


When you define variables in a separate file, you can define macros that span multiple lines. Let’s assume you have a set of application specific preparation steps. Create your steps in a separate file e.g. mysteps.spec Once you do that you can define your macro e.g. Now you have a macro “prepsteps” that can be used in any step after it’s defined. Here’s how you can use it in the “prep” phase e.g.

%prep
%prepsteps


Let’s pull some of this together with a simple example spec file:

$> rpmdev-newspec ex.spec
$> cat ex.spec
Name:           ex
Version:      
Release:        1%{?dist}
Summary:      
Group:        
License:      
URL:          
Source0:      
BuildRequires:
Requires:     
%description
 
%prep
%setup -q
 
%build
%configure
make %{?_smp_mflags}
 
%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT
 
%clean
rm -rf $RPM_BUILD_ROOT
 
%files
%defattr(-,root,root,-)
%doc
%changelog


If you run that with rpmbuild you’re going to get your first taste of errors. Since we’re playing around, let’s just try to run the prep phase.

$> rpmbuild -bp ex.spec
error: line 2: Empty tag: Version:


Unfortunately, rpmbuild only reports the first error and not the rest. Here is what it looks like if we fill it out some more:

Name:           ex
Version:        1.0.0
Release:        1%{?dist}
Summary:        This is an example
Group:          Example group
License:        GNU
URL:            http://example
Source0:        ex-%{version}.tar.gz
%description
 
%prep
%setup -q
 
%build
%configure
make %{?_smp_mflags}
 
%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT
 
%clean
rm -rf $RPM_BUILD_ROOT
 
%files
%defattr(-,root,root,-)
%doc
%changelog


Building RPMS: http://www.rpm.org/max-rpm/index.html 
Building RPMS: https://fedoraproject.org/wiki/How_to_create_an_RPM_package 
Naming guidelines: http://fedoraproject.org/wiki/Packaging:NamingGuidelines

Friday, October 30, 2015

Consul: Adding TLS to Consul using Self Signed Certificates

I'm currently working on setting up TLS for Consul. As of this writing, I'm still in the experimentation/set-up phase, but we plan to roll consul out into production with TLS support. So, this document may get updated but I wanted to capture what I had to do while it's fresh in my mind.

Consul's documents are a little light on specifics, which made this endeavor more difficult than I anticipated. I will post links at the bottom of this article. The following steps were used to create a self signed certificate on Centos 6.6. 

Make a directory to hold our files, create the certificate authority (ca) conf file, seed our index and create a cert index file:


> mkdir -p /opt/consul/ssl
> cat << EOF > /opt/consul/ssl/demo.conf
[ ca ]
default_ca = demo

[ crl_ext ]
# issuerAltName=issuer:copy  #this would copy the issuer name to altname
authorityKeyIdentifier=keyid:always

[ demo ]
new_certs_dir = /tmp
unique_subject = no
certificate = /opt/consul/ssl/demo-root.cer
database = /opt/consul/ssl/certindex
private_key = /opt/consul/ssl/privkey.pem
serial = /opt/consul/ssl/serial
default_days = 365
default_md = sha1
policy = demo_policy
x509_extensions = demo_extensions

[ demo_policy ]
commonName = supplied
stateOrProvinceName = supplied
countryName = supplied
emailAddress = optional
organizationName = supplied
organizationalUnitName = optional

[ demo_extensions ]
basicConstraints = CA:false
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always
keyUsage = digitalSignature,keyEncipherment
extendedKeyUsage = serverAuth,clientAuth
crlDistributionPoints = URI:http://path.to.crl/demo.crl
EOF

> touch /opt/consul/ssl/certindex
> echo 000a > /opt/consul/ssl/serial
> cd /opt/consul/ssl

NOTE: You may need this step to ensure the certs work with Consul. Edit the /etc/pki/tls/openssl.cnf file and add the following:

extendedKeyUsage=serverAuth,clientAuth

Below I have one way to make the change. Read more about extended key usage here: https://www.openssl.org/docs/manmaster/apps/x509v3_config.html#extended_key_usage_

> cp /etc/pki/tls/openssl.cnf  /etc/pki/tls/openssl.cnf.bak
> sed -i"" 's|# extendedKeyUsage = critical,timeStamping|extendedKeyUsage=serverAuth,clientAuth|' /etc/pki/tls/openssl.cnf


Generate the root certificate:

> openssl req -newkey rsa:2048 -days 3650 -x509 -nodes -out /opt/consul/ssl/demo-root.cer -keyout /opt/consul/ssl/private.pem 
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:New York
Locality Name (eg, city) [Default City]:New York
Organization Name (eg, company) [Default Company Ltd]:Demo Company
Organizational Unit Name (eg, section) []:Demo
Common Name (eg, your name or your server's hostname) []:
Email Address []:


Consul want's the certs and the servers to be server.<data center>.consul - just adjust the request below as I used dc1 as my datacenter. Generate a certificate signer request (csr):

> openssl req -newkey rsa:1024 -nodes -out /opt/consul/ssl/server.csr -keyout /opt/consul/ssl/server.key
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:New-York
Locality Name (eg, city) []:New York
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Demo Company
Organizational Unit Name (eg, section) []:Demo
Common Name (e.g. server FQDN or YOUR name) []:server.dc1.consul
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:


Generate the self signed cert:

> openssl ca -batch -config /opt/consul/ssl/demo.conf -notext -in /opt/consul/ssl/server.csr -out /opt/consul/ssl/server.cer

To verify your certificate use the following command and make sure the "X509v3 Extended Key Usage" matches:

> openssl x509 -noout -text -in /opt/consul/ssl/server.cer
.....
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
.....

Now you can configure your consul server to use the self signed certs. These lines were take out of my consul.json file:

    "ca_file": "/opt/consul/ssl/demo-root.cer",
    "cert_file": "/opt/consul/ssl/server.cer",
    "key_file": "/opt/consul/ssl/server.key",

On your agents, you're going to need to specify the "ca_file" and set "verify_outgoing":true in your consul configs. 

If you get errors about trusting the signing authority, you will need to trust the demo-root.cer. To trust the root certificate on your server(s) do the following:

1. Install the ca-certificates package
2. Enable the dynamic CA configuration feature
3. Add it as a new file to /etc/pki/ca-trust/source/anchors/:
4. Use command:

> yum install -y ca-certificates
> update-ca-trust enable
> cp /opt/consul/ssl/demo-root.cer /etc/pki/ca-trust/source/anchors/
> update-ca-trust extract

Here are the documents I had to read:

Wednesday, October 14, 2015

Generating a public key for SSH using your private RSA key

In order to ssh onto a server using public private key pairs, you need a specific type of public key. If you have the private key you can generate the public to install into the ~/.ssh/authorized_keys file with the following

echo "actual private key data" > private
chmod 600 private
ssh-keygen -y -f private

You can generate a public key with the following

 openssl rsa -in private.pem -pubout > public

But it won't work for using ssh e.g. ssh -i private

Tuesday, June 2, 2015

Changing commit logs in git

It sounds like something you shouldn't do, but sometimes you may want to adjust who made a commit. Maybe you did a commit on a vagrant box or maybe you fat fingered your name or email address while typing too fast. To change the committer I found this handy
#!/bin/sh
 
git filter-branch --env-filter '

OLD_EMAIL="bad@emailaddress"
CORRECT_NAME="Russell Simpkins"
CORRECT_EMAIL="russellsimpkins@real-domain"

if [ "$GIT_COMMITTER_EMAIL" = "$OLD_EMAIL" ]
then
    export GIT_COMMITTER_NAME="$CORRECT_NAME"
    export GIT_COMMITTER_EMAIL="$CORRECT_EMAIL"
fi
if [ "$GIT_AUTHOR_EMAIL" = "$OLD_EMAIL" ]
then
    export GIT_AUTHOR_NAME="$CORRECT_NAME"
    export GIT_AUTHOR_EMAIL="$CORRECT_EMAIL"
fi
' --tag-name-filter cat -- --branches --tags

Simply adjust the OLD_EMAIL, CORRECT_NAME and CORRECT_EMAIL and stuff that into a bash script. Then issue a git push --force

Thursday, May 28, 2015

Resizing an ext4, ebs volume

You use lsblk and see your EBS volume is the right size, but running df -h shows the device is smaller. To fix this, the command to use is resize2fs 

resize2fs /dev/xvdf

While you can do it on a mounted device, these things are often better done when it's unmounted, just to be safe.

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Storage_Administration_Guide/ext4grow.html

Monday, April 13, 2015

Fun with GPG


I once had a desire to create a team GPG key that I could use for signing RPMs. I've moved in a different direction, but I want to capture the steps in case I decide to use this again in the future.

You can import a private GPG with the following:

gpg --allow-secret-key-import --import private.key.file
gpg --list-keys
gpg --edit-key <ID> 

Once you run --edit-key you're able to trust the key. Execute **trust** and choose level **5**

With that done, you can decrypt using the key - assuming you know the password.

gpg -d -u "name <email>" encrypted.file.gpg > outputfile


To encrypt for the team key to unlock:

gpg -se -r "name <email>" -u "name <email>" encrypted.file

Thursday, February 5, 2015

Port Forwarding on Mac OSX

If your running vagrant and you're forwarding traffic to vagrant over 8080, but you really prefer to hit port 80, you can use Mac's pfctl function. 

Here's a couple of links that you might find helpful
https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man8/pfctl.8.html
http://krypted.com/mac-os-x/a-cheat-sheet-for-using-pf-in-os-x-lion-and-up/

I was reading this article http://salvatore.garbesi.com/vagrant-port-forwarding-on-mac/ and it suggested adding a vagrant plugin, but it's a ruby gem. Hard to imagine, but the gem failed to install.

You can still implement port forwarding. Create a pfctl.conf file in your vagrant folder:

echo "rdr pass on lo0 inet proto tcp from any to 127.0.0.1 port 80 -> 127.0.0.1 port 8080
rdr pass on lo0 inet proto tcp from any to 127.0.0.1 port 443 -> 127.0.0.1 port 8443" > pfctl.conf

To run this, you first need to enable pfctl

pfctl -e -f pfctl.conf

Once you enable the firewall rules, you can hit your vagrant box by going against localhost. 

To disable pfctl: 

pfctl -d