Thursday, October 22, 2009

Slowing brute force hackers

I now use denyhosts, which is available via yum, but a while back I used the solution below.
I found this post on novell.com for a script to stop hackers. I altered mine and think this may be helpful to others.

Here is the original post: http://www.novell.com/coolsolutions/trench/16341.html

I installed this script on the crontab. It effectivly blocks specific hack attempts, in a semi-permanent way. Each night, logrotate wipes out the log of the hacks and stops blocking the ip. In one light this is good. It blocks the offending IP log enough to prevent harm, and later opens it back up since, most likely, it's a DHCP address anyway.

The next step for me is to run a script pre-logrotate to save the offending IP address for historical analysis and potential abuse reports.


Here are my changes:
#!/bin/bash
# AUTHOR: By Chander Ganesan

LAST_IP=0.0.0.0
COUNT=1

# Set MAXCOUNT to the maximum failures allowed before blacklisting
MAXCOUNT=5

#
# The three lines below put the leading lines in /etc/hosts.allow
# Note: This script overwrites the entire /etc/hosts.allow file.
#

echo '
# /etc/hosts.deny
# See "man tcpd" and "man 5 hosts_access" as well as /etc/hosts.allow
# for a detailed description.
http-rman : ALL EXCEPT LOCAL' > /etc/hosts.deny

#
# Scan the /var/log/messages file for failed login attempts via ssh.
# Parse out the IP address, and count the failure occurances from that IP
# If the IP fails more than 5 times - deny further access
#
# RSS: Changed grep to search for "Failed password". Added second sed command
for IP in `/bin/grep sshd /var/log/secure|/bin/grep "Failed password"|/bin/sed
's/^.*from :*[a-z]*://'|/bin/sed 's/ .*//'` 0.0.0.0; do
if [ ${LAST_IP} == ${IP} ]; then
     let COUNT=${COUNT}+1
else
     if [ ${COUNT} -ge ${MAXCOUNT} ]; then
        echo "ALL: ${LAST_IP}" >> /etc/hosts.deny
     fi
     LAST_IP=${IP}
     COUNT=1
fi
done

Generating a new CSR to get your shiny new SSL certificate

Adding a new SSL client involves creating a key and certificate signing request. You can always get these instructions from your certificate company, ours is Geo Trust, here is their instructions. http://www.geotrust.com/resources/csr/apache2.htm


# create key w/out password
openssl genrsa -out domainname.key 2048

# view key:
openssl rsa -noout -text -in domainname.key

# gen  the CSR:
openssl req -new -key domainname.key -out domainname.csr

Turning man pages to PDF files

I really don't like reading on the terminal. I just realized that I can create PDF files of my man pages, and it's really easy.

Normally you use man like so: man bash
but if you want to turn that into a pdf file, its really easy. Instead type:

man -t bash > bash.manpage.ps
ps2pdf bash.manpage.ps bash.manpage.pdf

and that is it!!

Regex pattern for stripping Javadocs

I recently had a need to remove all Javadoc comments from some code. I use JEdit a lot, and it, like others allows you to use RE's to search through text. The RE for ridding your code of all java comments is 
[/**].*

XSLT outputing Carriage return/line feed

A couple of years ago, I was trying to use Open Office writer to convert documents to media wiki format using this XSLT file I found. It worked ok, but there was more full featured XSLT here
Anyway, I found that the first XSLT was not putting out the Carriage return linefeed or any linefeed out in it's output. I discovered that I could put &#xA; into the <xsl:value-of select="''"/> statement, i.e. <xsl:value-of select="'&#xA;==='"/> which, for media wiki is the starting === with a line feed in there.

The activasoft way was a bit easier <xsl:text>
===
</xsl:text>

I think its good to know both ways. I know I had to tackle this before, which is why I post this reminder now.

Getting FTP to work through IP tables

Say you're like me, and you've recently configured a server with vsftpd, and you're pretty sure you have all your firewall rules done right, and you can connect to the ftp server, but you just can't seem to list directories. You get a LIST command 500 error and your annoyed. If so, then this might help. If you use NAT on your server with vsftp you must allow ftp natting.

You need to edit this file:
/etc/sysconfig/iptables-config

Add follow line:
IPTABLES_MODULES="ip_nat_ftp"

And then restart your iptables and vsftpd and you're off and running.

Yum repo for more up to date Dovecot

While working with dovecot on centos 5 back in 2007, I discovered that the latest dovecot was not part of the yum updates, and fortunatly there are more up to date rpms here: 
http://atrpms.net/dist/el5/

JBoss and file handles on RedHat

I discovered that my jboss 4.0.5-GA instance was running out of file handles on subsequent war deployments, so I had to set find a way to make the server more robust. I did a little googling around on ulimit. At the time, I was running JBoss as root and finally decided to break down and create a jboss account. I then config /etc/security/limits.conf; borrowing from an article on setting up oracle for more file handles. I added: 
jboss soft nofile 4096
jboss hard nofile 63536 
I created a jboss account without a password and added jboss/bin/init.id/jboss as a sym link to boss's $JBOSS_HOME/bin/jboss_init_redhat.sh. I modified the start script to put the log file to /var/log/jboss.log and opted to set the default server for my server. You do have to add a couple of lines for chkconfig to work right: 
#!/bin/sh
#
# jboss Startup script for the JBoss J2EE App Server
#
# chkconfig: - 85 15
# description: JBoss is a J2EE app server
#
# processname: jboss 
Then you can add to the init tab with:

chkconfig --add jboss
chkconfig --level 5 jboss on

How to get more network stats

A little while back, I was load testing a server and needed a way to verify the cards on the server and just see what was going on for network traffic. I found these commands helpful:

sudo mii-tool
ethtool eth0
dmesg | grep eth

For other networking throughput tests you can install iftop and bmw-ng

sudo su -c 'rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm'
sudo yum install -y iftop bwm-ng

then you can run

iftop
iftop -i eth0
bwm-ng

Search around if you want some more posts, these are just my crib notes. :-)

How to find process IDs listening to port xxx

If you need to figure out which process ids are listening to a given port by id, and netstat isn't working for you, this will work:

lsof -i :80 | awk '{print $2}'|grep -e [0-9]

Wednesday, October 21, 2009

Fun with SSL certs and Apache

I recently ran into a site where I had to enter the pass phrase to restart Apache, but I wasn't 100% certain of the password and didn't want to bring down the live site if I didn't have to. I did a little searching and found this useful trick to test with openssl e.g.

openssl s_server -cert crt/public.crt -key private/private.key -www

When you run that, if your key was created with a pass phrase, you will get prompted for the password and you can test to make sure you have the right certificate password, without having to worry about killing the site.

If you really don't want to have to enter the pass phrase and you don't have the time or care to re-do the certificate, you can get away with using expect. Here is a sample script

#!/bin/bash


password=secret
scriptname=/usr/apache/bin/apachectl
arg1=start
timeout=-1
# now connect to remote UNIX box (ipaddr) with given script to execute
spawn $scriptname $arg1
match_max 100000
# Look for password prompt
expect "*?ass phrase:*"
# Send password aka $password
send -- "$password\r"
# send blank line (\r) to make sure we get back to gui
send -- "\r"
expect eof

You can use expect for other programs. The only think you might have to work on is the expect part where you have to give expect the term to look for.

Monday, October 19, 2009

Adding Jalopy to Maven and clean things up


Recently, I had the opportunity to work with web services and Apache Axis tools for generating java code from WSDL files to use UPS's Tradability web services. The generator created a lot of code that was poorly formatted. Given that I hate doing repetitive tasks, and I was already building in maven, I decided to see whose got a code formatter for maven. That way I could generate code and format code with just a few typed commnds.

Jalopy came to my rescue. To add their plugin, add the following to your pom.xml file:


<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>jalopy-maven-plugin</artifactId>
  <version>1.0-alpha-1</version>
  <configuration>
    <fileFormat>UNIX</fileFormat>
    <convention>src/main/resources/jalopy/jalopy-settings.xml</convention>
  </configuration>
</plugin>


Now, you don't have to use your own config file, but I recommend it if you want Sun coding conventions. This isn't perfect, I'm sure, but it's close enough for my needs:

<?xml version="1.0" encoding="UTF-8"?>
<jalopy>
    <general>
        <compliance>
            <version>13</version>
        </compliance>
        <style>
            <description>Java Style</description>
            <name>Maven2</name>
        </style>
    </general>
    <inspector>
        <enable>false</enable>
        <naming>
            <classes>
                <abstract>[A-Z][a-zA-Z0-9]+</abstract>
                <general>[A-Z][a-zA-Z0-9]+</general>
            </classes>
            <fields>
                <default>[a-z][\w]+</default>
                <defaultStatic>[a-z][\w]+</defaultStatic>
                <defaultStaticFinal>[a-zA-Z][\w]+</defaultStaticFinal>
                <private>[a-z][\w]+</private>
                <privateStatic>[a-z][\w]+</privateStatic>
                <privateStaticFinal>[a-zA-Z][\w]+</privateStaticFinal>
                <protected>[a-z][\w]+</protected>
                <protectedStatic>[a-z][\w]+</protectedStatic>
                <protectedStaticFinal>[a-zA-Z][\w]+</protectedStaticFinal>
                <public>[a-z][\w]+</public>
                <publicStatic>[a-z][\w]+</publicStatic>
                <publicStaticFinal>[a-zA-Z][\w]+</publicStaticFinal>
            </fields>
            <interfaces>[A-Z][a-zA-Z0-9]+</interfaces>
            <labels>\w+</labels>
            <methods>
                <default>[a-z][\w]+</default>
                <defaultStatic>[a-z][\w]+</defaultStatic>
                <defaultStaticFinal>[a-z][\w]+</defaultStaticFinal>
                <private>[a-z][\w]+</private>
                <privateStatic>[a-z][\w]+</privateStatic>
                <privateStaticFinal>[a-z][\w]+</privateStaticFinal>
                <protected>[a-z][\w]+</protected>
                <protectedStatic>[a-z][\w]+</protectedStatic>
                <protectedStaticFinal>[a-z][\w]+</protectedStaticFinal>
                <public>[a-z][\w]+</public>
                <publicStatic>[a-z][\w]+</publicStatic>
                <publicStaticFinal>[a-z][\w]+</publicStaticFinal>
            </methods>
            <packages>[a-z]+(?:\.[a-z]+)*</packages>
            <parameters>
                <default>[a-z][\w]+</default>
                <final>[a-z][\w]+</final>
            </parameters>
            <variables>[a-z][\w]*</variables>
        </naming>
        <tips>
            <adhereToNamingConvention>false</adhereToNamingConvention>
            <alwaysOverrideHashCode>false</alwaysOverrideHashCode>
            <avoidThreadGroups>false</avoidThreadGroups>
            <declareCollectionComment>false</declareCollectionComment>
            <dontIgnoreExceptions>false</dontIgnoreExceptions>
            <dontSubstituteObjectEquals>false</dontSubstituteObjectEquals>
            <neverDeclareException>false</neverDeclareException>
            <neverDeclareThrowable>false</neverDeclareThrowable>
            <neverInvokeWaitOutsideLoop>false</neverInvokeWaitOutsideLoop>
            <neverReturnZeroArrays>false</neverReturnZeroArrays>
            <neverUseEmptyFinally>false</neverUseEmptyFinally>
            <obeyContractEquals>false</obeyContractEquals>
            <overrideToString>false</overrideToString>
            <referToObjectsByInterface>false</referToObjectsByInterface>
            <replaceStructureWithClass>false</replaceStructureWithClass>
            <stringLiterallI18n>false</stringLiterallI18n>
            <useInterfaceOnlyForTypes>false</useInterfaceOnlyForTypes>
            <wrongCollectionComment>false</wrongCollectionComment>
        </tips>
    </inspector>
    <internal>
        <version>6</version>
    </internal>
    <messages>
        <priority>
            <general>30000</general>
            <parser>30000</parser>
            <parserJavadoc>30000</parserJavadoc>
            <printer>30000</printer>
            <printerJavadoc>30000</printerJavadoc>
            <transform>30000</transform>
        </priority>
        <showErrorStackTrace>true</showErrorStackTrace>
    </messages>
    <misc>
        <threadCount>2</threadCount>
    </misc>
    <printer>
        <alignment>
            <methodCallChain>false</methodCallChain>
            <parameterMethodDeclaration>false</parameterMethodDeclaration>
            <ternaryOperator>true</ternaryOperator>
            <variableAssignment>false</variableAssignment>
            <variableIdentifier>false</variableIdentifier>
        </alignment>
        <backup>
            <directory>bak</directory>
            <level>0</level>
        </backup>
        <blanklines>
            <after>
                <block>1</block>
                <braceLeft>0</braceLeft>
                <class>1</class>
                <declaration>0</declaration>
                <footer>1</footer>
                <header>0</header>
                <interface>1</interface>
                <lastImport>1</lastImport>
                <method>0</method>
                <package>0</package>
            </after>
            <before>
                <block>1</block>
                <braceRight>0</braceRight>
                <caseBlock>0</caseBlock>
                <comment>
                    <javadoc>1</javadoc>
                    <multiline>1</multiline>
                    <singleline>1</singleline>
                </comment>
                <controlStatement>0</controlStatement>
                <declaration>0</declaration>
                <footer>0</footer>
                <header>0</header>
            </before>
            <keepUpTo>1</keepUpTo>
        </blanklines>
        <braces>
            <empty>
                <cuddle>false</cuddle>
                <insertStatement>false</insertStatement>
            </empty>
            <insert>
                <dowhile>true</dowhile>
                <for>true</for>
                <ifelse>true</ifelse>
                <while>true</while>
            </insert>
            <remove>
                <block>true</block>
                <dowhile>false</dowhile>
                <for>false</for>
                <ifelse>false</ifelse>
                <while>false</while>
            </remove>
            <treatDifferent>
                <methodClass>false</methodClass>
                <methodClassIfWrapped>false</methodClassIfWrapped>
            </treatDifferent>
        </braces>
        <chunks>
            <blanklines>false</blanklines>
            <comments>true</comments>
        </chunks>
        <comments>
            <format>
                <multiline>true</multiline>
            </format>
            <javadoc>
                <check>
                    <innerclass>false</innerclass>
                    <tags>false</tags>
                    <throwsTags>true</throwsTags>
                </check>
                <fieldsShort>false</fieldsShort>
                <generate>
                    <class>1</class>
                    <constructor>0</constructor>
                    <field>0</field>
                    <method>0</method>
                </generate>
                <parseComments>true</parseComments>
                <tags>
                    <in-line />
                    <standard />
                </tags>
                <templates>
                    <class>/**| * DOCUMENT ME!| *| * @author $author$| * @version $Revision$| */</class>
                    <constructor>
                        <bottom> */</bottom>
                        <exception> * @throws $exceptionType$ DOCUMENT ME!</exception>
                        <param> * @param $paramType$ DOCUMENT ME!</param>
                        <top>/**| * Creates a new $objectType$ object.</top>
                    </constructor>
                    <interface>/**| * DOCUMENT ME!| *| * @author $author$| * @version $Revision$| */</interface>
                    <method>
                        <bottom> */</bottom>
                        <exception> * @throws $exceptionType$ DOCUMENT ME!</exception>
                        <param> * @param $paramType$ DOCUMENT ME!</param>
                        <return> * @return DOCUMENT ME!</return>
                        <top>/**| * DOCUMENT ME!</top>
                    </method>
                </templates>
            </javadoc>
            <remove>
                <javadoc>false</javadoc>
                <multiline>false</multiline>
                <singleline>false</singleline>
            </remove>
            <separator>
                <fillCharacter>-</fillCharacter>
                <insert>false</insert>
                <insertRecursive>false</insertRecursive>
                <text>
                    <class>Inner Classes</class>
                    <constructor>Constructors</constructor>
                    <field>Instance fields</field>
                    <initializer>Instance initializers</initializer>
                    <interface>Inner Interfaces</interface>
                    <method>Methods</method>
                    <static>Static fields/initializers</static>
                </text>
            </separator>
        </comments>
        <environment />
        <footer>
            <keys />
            <smartMode>0</smartMode>
            <use>false</use>
        </footer>
        <header>
            <keys>Tha Apache Software License</keys>
            <smartMode>5</smartMode>
            <text>/*| * Copyright 2001-2004 The Apache Software Foundation.| *| * Licensed under the Apache License, Version 2.0 (the "License");| * you may not use this file except in compliance with the License.| * You may obtain a copy of the License at| *| *      http://www.apache.org/licenses/LICENSE-2.0| *| * Unless required by applicable law or agreed to in writing, software| * distributed under the License is distributed on an "AS IS" BASIS,| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.| * See the License for the specific language governing permissions and| * limitations under the License.| */</text>
            <use>false</use>
        </header>
        <history>
            <policy>disabled</policy>
        </history>
        <imports>
            <grouping>
                <defaultDepth>3</defaultDepth>
                <packages>*:0|gnu:2|java:1|javax:1</packages>
            </grouping>
            <policy>disabled</policy>
            <sort>true</sort>
        </imports>
        <indentation>
            <caseFromSwitch>true</caseFromSwitch>
            <continuation>
                <block>false</block>
                <operator>false</operator>
            </continuation>
            <firstColumnComments>false</firstColumnComments>
            <label>false</label>
            <policy>
                <deep>true</deep>
            </policy>
            <sizes>
                <braceCuddled>1</braceCuddled>
                <braceLeft>1</braceLeft>
                <braceRight>0</braceRight>
                <braceRightAfter>1</braceRightAfter>
                <continuation>4</continuation>
                <deep>55</deep>
                <extends>-1</extends>
                <general>4</general>
                <implements>-1</implements>
                <leading>0</leading>
                <tabs>8</tabs>
                <throws>-1</throws>
                <trailingComment>1</trailingComment>
            </sizes>
            <tabs>
                <enable>false</enable>
                <onlyLeading>false</onlyLeading>
            </tabs>
        </indentation>
        <misc>
            <arrayBracketsAfterIdent>false</arrayBracketsAfterIdent>
            <forceFormatting>false</forceFormatting>
            <insertExpressionParentheses>true</insertExpressionParentheses>
            <insertLoggingConditional>false</insertLoggingConditional>
            <insertTrailingNewline>true</insertTrailingNewline>
            <insertUID>false</insertUID>
        </misc>
        <sorting>
            <declaration>
                <class>false</class>
                <constructor>false</constructor>
                <enable>false</enable>
                <interface>false</interface>
                <method>false</method>
                <order>static|field|initializer|constructor|method|interface|class</order>
                <variable>false</variable>
            </declaration>
            <modifier>
                <enable>false</enable>
                <order>public|protected|private|abstract|static|final|synchronized|transient|volatile|native|strictfp</order>
            </modifier>
        </sorting>
        <whitespace>
            <after>
                <comma>true</comma>
                <semicolon>true</semicolon>
                <typeCast>true</typeCast>
            </after>
            <before>
                <braces>false</braces>
                <brackets>false</brackets>
                <bracketsTypes>false</bracketsTypes>
                <caseColon>false</caseColon>
                <operator>
                    <not>true</not>
                </operator>
                <parentheses>
                    <methodCall>false</methodCall>
                    <methodDeclaration>false</methodDeclaration>
                    <statement>false</statement>
                </parentheses>
            </before>
            <padding>
                <braces>false</braces>
                <brackets>false</brackets>
                <operator>
                    <assignment>true</assignment>
                    <bitwise>true</bitwise>
                    <logical>true</logical>
                    <mathematical>true</mathematical>
                    <relational>true</relational>
                    <shift>true</shift>
                </operator>
                <parenthesis>false</parenthesis>
                <typeCast>false</typeCast>
            </padding>
        </whitespace>
        <wrapping>
            <always>
                <after>
                    <arrayElement>0</arrayElement>
                    <braceRight>false</braceRight>
                    <extendsTypes>false</extendsTypes>
                    <implementsTypes>false</implementsTypes>
                    <label>true</label>
                    <methodCallChained>false</methodCallChained>
                    <ternaryOperator>
                        <first>false</first>
                        <second>false</second>
                    </ternaryOperator>
                    <throwsTypes>false</throwsTypes>
                </after>
                <before>
                    <braceLeft>false</braceLeft>
                    <extends>true</extends>
                    <implements>true</implements>
                    <throws>true</throws>
                </before>
                <parameter>
                    <methodCall>false</methodCall>
                    <methodCallNested>true</methodCallNested>
                    <methodDeclaration>false</methodDeclaration>
                </parameter>
            </always>
            <general>
                <beforeOperator>false</beforeOperator>
                <enable>true</enable>
                <lineLength>140</lineLength>
            </general>
            <ondemand>
                <after>
                    <assignment>true</assignment>
                    <leftParenthesis>false</leftParenthesis>
                    <parameter>true</parameter>
                    <types>
                        <extends>true</extends>
                        <implements>true</implements>
                        <throws>true</throws>
                    </types>
                </after>
                <before>
                    <rightParenthesis>false</rightParenthesis>
                </before>
                <groupingParentheses>true</groupingParentheses>
            </ondemand>
        </wrapping>
    </printer>
</jalopy>

Friday, October 16, 2009

Dealing with extended characters in bash

This week I was writing a script that was searching for missing data in SGML files. I ran into issues with data that had extended characters in it. I did a little digging and discovered that iconv could come the rescue. The man pages let you specify the input format with -f, but I found that I had better luck leaving out -f. Here is what I used:

iconv -t UTF-8 --byte-subst="&#x%X;" target-file| grep "term" 
This may not be perfect, but I found this useful enough for my purposes. If this doesn't work, don't forget to try specifying -f.

...

Meta key in emacs on osx 10.6.1

I recently decided to get back to using emacs again and was very frustrated with the meta key. I searched and searched around trying to find something better than using [escape] key as meta. Eventually it dawned on me to look at my terminal preferences where I discovered the secret. There is a box you can select "use option as meta" that has made my life easier. I posted this for anyone else on the wiki: emacs wiki just in case anyone else is interested.

The fix above worked well for terminal emacs. For carbon emacs, you may have to add the following to your init.el file:

(setq mac-option-modifier 'meta)

as per: this post