jabberd14 for testing with jira

I'm working on getting a previously developed Jira IM library JIMI (which leveraged the work done in Yet Another IM Plugin) integrated with JEMH. The merge was always on the cards but until JEMH got bright enough to enable comment processing, the usability wasn't there. Now is the time.

With IM support, and an ability to modify issues, it becomes a whole new ballgame, so the old JIMI code is getting the dust blown off.

Current IM features include:

• Jira configuration for Jabber
• Additional authentication check, including captcha support
• access to system info, eg memory usage
• dynamic execution of arbitrary uploaded beanshell scripts. hmmmm.
  

Im thinking of future-features such as:

• watch a project
• watch a component
• watch an issue
• watch a user
• filters enabling matching of subject/comment text (aking to xyz was mentioned...)
• of course, modifying issues, eg reassign, change priority, resolve, start progress etc etc.

all doable, in time, question is, what can I get working in the next week :)

Jira, SSL and Gmail, what a faff!

I had cause today to verify correct operation of Jira with Gmail with JEMH. I had enough fun to want to write it down for future reference.

1. So Gmail needs SSL for POP and SMTP
The problem is that in order for Jira (Java actually) to setup even a single SSL connection, it needs to be able to trust the source.

Following the Connecting to SSL services page I recalled the nitty gritty of extracting the Gmail security cert, loading into the JRE cacerts file I remembered. Really usefully I was happy to the the old SSLPoke.class file I wrote to do testing ages ago still available, ironically able to use it to test my setup today! With a JRE now able to talk SSL to gmail, time to setup Jira.

2. Setup Jira
a) POP
Adding a popserver entry, with simple details, host as pop.gmail.com and valid username/password. The SSL bit comes in when setting up the handler
When adding a POP handler, ensure the gmail popserver is selected, the handler has a Uses SSL option, which needs setting true, and the port should be 995.

b) SMTP
Followed Configuring JIRA to send SMTP mail , so modified the config/server.xml file, all seemed fine, but this is where I hit a few snags, the most irritating was "java:comp/env....." not found. Turned out this was due to a space at the front of the " java:", gah, my kingdom for a trim().

I found JRA-12180 , so moved the two jars (mail / activation ) from atlassian-jira/WEB-INF/lib into common/lib. Q: Why isnt this already done as part of the standalone dist? huh.

And yet, it still didn't work. Hmm. So found an Atlassian knowledge base article
that indicated the "mail.smptp.socketFactoryClass" entry in the server.xml file that the earlier page instructed to be added.

Alls well that ends well. Gmail now working as SMTP/POP server over SSL,.

Exterminating debian packages that wont uninstall

Darn it,
you've install shiny package X that didn't come through the relatively QA'd Ubuntu package archive, you don't want it anymore and try and uninstall, and it all goes south. It doesn't uninstall cleanly and every 10minutes you get an update window telling you that only a partial update is posssible.

I ignored this for a few days but my annoyance grew. In my case, the package in question was db2, which Ive kind of moved away from to Postgres due to simple 'just work dammit' reasons.

Removing with the various dpg optons had no effect. Surgery required.

Scalpel in hand and a probable guess that the dpk 'installed packages' file was in /lib somewhere, I did the following:

find /var -exec grep -H db2exc {} \;

This little beuty scans every file for a substring match and prints the matching filename / line. bingo

Sure enough, the file was /var/lib/dpkg/status, a flick of the scalpel and db2exc was history.

I could have done it properly and fixed the uninstall script, but I was feeling ruthless, one of those rm -rf moments.

Automatically resize the 'family pics' collection for a photo frame

I have a C109D 9" Digital Photo frame, which has a 2GB SD flash card. I thought that would have been enough, but I hit the limit over the weekend. This spurred to to finally figure out how to bulk compress images, and annoyingly, not to just convert them but to convert them in a different place so the originals are available for printing.

Resizing several Gb of images is a tedious job, fortunately there are some nifty tools out there to help. At the core of the process is the imagemagick convert program, the rest is syntactic script-sugar.

I wrote two scripts to do this, the first makes the first set of root folders in the target directory and calls the second to do the heavy lifting of per-root folder processing.

A bug in the photo-frame code meant it hung if it encountered an empty directory (took a while to figure that out), so as a post-process, after the compression process has completed, script1 finds all folders with no files, and adds a default as a workaround.

SCRIPT 1 - create.sh
My top level folders were simply the year, eg 2001 etc.

#!/bin/bash
PICS_HOME=/srv/FamilyPics
COMPRESS_HOME=/srv/compressedPics
cd $COMPRESS_HOME
#rm -rf $COMPRESS_HOME/*
for aDir in `find $PICS_HOME -maxdepth 1 -name '200*' -type d`
do
BASEDIR=`basename $aDir`
#echo Making top level folder: $COMPRESS_HOME/$BASEDIR
mkdir -p $COMPRESS_HOME/$BASEDIR
cd $aDir
find . -iname "*.jpg" -exec $PICS_HOME/compressFile.sh $PICS_HOME $COMPRESS_HOME $aDir "{}" \;
done

#Now fixup empty dirs that break the digital photoframe
PLACEHOLDER="$COMPRESS_HOME/2002/someplace/placeholder.jpg"
DIRS=`find $COMPRESS_HOME -type d`
for aDir in $DIRS
do
FILE_COUNT=`find $aDir -maxdepth 1 -type f |wc -l`
if [ $FILE_COUNT -eq 0 ]; then
echo need to fix dir $aDir : $FILE_COUNT
cp $PLACEHOLDER $aDir/.
fi
done

SCRIPT 2 - compressFile.sh
#!/bin/bash
PICS_HOME="$1"
COMPRESS_HOME="$2"
aDir="$3"
BASEDIR=`basename "$aDir"`
aFile="$4"

#echo "picsHome=[$PICS_HOME] compressHome=[$COMPRESS_HOME] aDir=[$aDir] baseDir=[$BASEDIR] aFile=[$aFile]"

FILENAME=`basename "$aFile"`
RELATIVE_PATH=`echo $aFile | sed -e 's/$FILENAME//g' |sed -e 's/ /-/g' | sed -e 's/\.\///g'`
OUTPUT_PATH="$COMPRESS_HOME/$BASEDIR/$RELATIVE_PATH"
OUTPUT_DIR=`dirname $OUTPUT_PATH`
#echo Making Folder path for output: $OUTPUT_DIR
mkdir -p $OUTPUT_DIR
# echo Compressing $BASEDIR/$aFile to $OUTPUT_PATH
cd `dirname $OUTPUT_PATH`
RAW_FILENAME="`basename $OUTPUT_PATH`"

if [ ! -f $OUTPUT_PATH ]; then
if [ -s "$PICS_HOME/$BASEDIR/$aFile" ]; then

cp "$PICS_HOME/$BASEDIR/$aFile" $OUTPUT_PATH
chmod +w $OUTPUT_PATH

# dd if=$PICS_HOME/jpg.hdr of="$RAW_FILENAME" bs=2 count=1
# echo RAW filename=$RAW_FILENAME
# echo Current dir `pwd`
# echo Compressing $OUTPUT_PATH
convert $RAW_FILENAME -encoding jpg -resize 640x480 -quality 86 -strip $RAW_FILENAME.tmp
rm "./$RAW_FILENAME"
mv "./$RAW_FILENAME.tmp" "$RAW_FILENAME"
OLDSIZE=`ls -l "$PICS_HOME/$BASEDIR/$aFile" | gawk 'FS=" " { print $5 }'`
NEWSIZE=`ls -l "$RAW_FILENAME" | gawk 'FS=" " { print $5 }'`
echo Compressed $OUTPUT_PATH from $OLDSIZE to $NEWSIZE
else
echo "File is empty: $PICS_HOME/$BASEDIR/$aFile"
fi
else
OLDSIZE=`ls -l "$PICS_HOME/$BASEDIR/$aFile" | gawk 'FS=" " { print $5 }'`
NEWSIZE=`ls -l "$RAW_FILENAME" | gawk 'FS=" " { print $5 }'`
echo Already Compressed $OUTPUT_PATH from $OLDSIZE to $NEWSIZE
fi

The result of this compression is that my 2Gb cart is now only 10% full rather than 100% full, so in theory, should be good for another 90 years, we'll probably have HD photo frames by then so the topic may become moot!

I hope these scripts will be useful fodder to anyone needing to do a similar thing.