killianfaughnan.com

CsvSQL 0.2.2 Released

Posted on July 16, 2012 in csvsql, perl by

The newest version of CsvSQL (0.2.2) can now be downloaded as either a tarball or zip archive:

csvsql-0.2.2.tgz

csvsql-0.2.2.zip

If you have and questions, suggestions, criticisms or comments please let me know as I would be interested in hearing them. And if you do decide to use this program for something, let me know how you get on!

CHANGELOG

This updated version includes the following fixes/changes:

  • Fixed issue whereby an entire column of blank values would cause an ‘uninitialized variable’ error
  • Fixed issue where a blank column heading would cause ‘uninitialized variable’ error.  Empty column headings now replaced by ‘Col#’ where # is the column index
  • Removed coloured printing as people most likely prefer their own colour schemes
  • Changed default display field separator from ‘|’ to ‘,’ (NOTE: this separator is only used when printing output to screen and not when reading input files – that separator is configured separately).  The ‘|’ makes for more readable ‘MySQL’ feel tables in a terminal, but ‘,’ is more useful, and probably more practical.
  • Added new parameter of ‘-f’ to allow the user to choose the displayed field separator

 

Burp proxy error: Received fatal alert: handshake_failure

Posted on February 19, 2012 in networking, security by

I came across this weird Burp issue recently where I keep receiving the following error when connecting to certain SSL sites (I suspect ones with no Secure SSL Renegotiation enabled?).  It looks to be a common intercepting proxy issue as ZAP proxy had the same issue when I tested it there.

  • Burp proxy error: Received fatal alert: handshake_failure

If you search the error you end up with a number of posts pointing towards this being a generic Java issue to do with the renegotiation of certs plus a few other things possible issues.  I tried allowing unsafe renegotiation both at the JVM level with “-Dsun.security.ssl.allowUnsafeRenegotiation=true” and in the SSL tab in Burp but no good.  I also tried every combination of SSL settings in Burp I could think of, but again no good.  In the end I decided to just circumvent Burp and have something else handle the SSL.

The below stunnel command (http://www.stunnel.org/ – available for both Linux + Windows) will start a daemon listening on your localhost and forward anything that connects to it to the target address over SSL.  Then in Burp just add an entry to the Hostname Resolution section (options->connections tab) pointing the domain you’re connecting to to localhost. (i.e. target.example.com – 127.0.0.1).  Ensure you’re proxying the connection through Burp and then in your browser open up http://target.example.com:443 (note the http, not https).

  • stunnel -cf -d 127.0.0.1:4443 -r target.example.com:443

Problem solved.  This way we’re using the OpenSSL SSL implementation and not the Java one.

 

Writing Better Perl

Posted on February 23, 2011 in perl by

I’ve recently started using a few tools to help clean up my Perl.  The first of these tools I use for standardised formatting of my Perl scripts.  The second I use to analyse my Perl source code.

Perltidy

The first of these tools is called perltidy, which is a perl script “indenter and reformatter”.  Essentially it allows you to keep a universal formatting standard across all your Perl scripts, and even better, across all Perl scripts written by others you work with too if you can convince them of the need!

Perltidy has many options, but I tend to stick to a pretty basic set of requirements:

perltidy -ce -b <file>.pl

The -ce flags above are short for –cuddled-else, which means we want to have else and elsif’s following immediately after the curly brace belonging to the previous block.

The -b tells perltidy to create a backup of the file as <file>.pl.bak, and modify the file in place.  The default behaviour is to writeout a new file to <file>.pl.tdy.

Perlcritic

For analysing my perl I use perlcritic.  The following extract is taken from the perlcritic man page:

“the executable front-end to the Perl::Critic engine, which attempts to identify awkward, hard to read, error-prone, or unconventional constructs in your code.  Most of the rules are based on Damian Conway’s book Perl Best Practices.”

OpsView check_snmp_runningproc Not Matching Process List

Posted on June 29, 2010 in linux, perl by

I was adding a few checks for oracle processes using check_snmp_runningproc and noticed a few problems. It seems that when the check returns a list of processes from a remote host it lowercases them all and attempts to strips out .e .ex and .exe extensions. The extensions used in the regular expression weren’t escaping the dot which resulted them in stripping out any character followed by an e, ex or exe.

In my case this meant it was matching against “orac” instead of oracle. I also noticed that the user input wasn’t undergoing the same translations as the returned processes (both the conversion to lower case and the stripping of extensions). I fixed this by cleaning up the regexp, and using an lc() function and the same regexp for user input. I submitted the patch which has been included in SVN. The most up to date version of this check is here:

http://svn.opsview.org/opsview/trunk/opsview-core/nagios-plugins/check_snmp_runningproc

Getting Your Ubuntu 8.10 Notification Back In 9.04

Posted on July 8, 2009 in linux by

If you miss the notification that used to be in the 8.10 notification area telling you new updates are available you can get it back by quite easily by issuing the following command.

gconftool -s --type bool /apps/update-notifier/auto_launch false

Then you just kill and restart update-notifier and you will have your icon for update-notifier back (to check it straight away, run: sudo apt-get update).

Note: This will also stop the update manager window from automatically popping up every time you run apt-get update which is reason enough to do it on its own.

Changing Your Tag’s Case In WordPress

Posted on July 7, 2009 in php by

Yet another pet hate of mine is things capitalised incorrectly. For me things like tags need to be either first letter uppercased of lowercases, and definitely not a mix of both. The problem with WordPress is that you unfortunately cannot change the case of a tag once it’s been used. This is because all your tags are put into a table the first time you use them, and from then on no matter what case you type a tag in, it will default to the capitalisation that is used in this table. The offending table is the wp-terms table.

This table contains a list of names and slugs (url friendly names) for all your tags. If you want to change the case of one of your tags you will need to do it here, either via console or phpMyAdmin or something else. Myself I went and just set all the names equal to the slug, and then changed the one or two tags I had with spaces in them back to something more rational. Whether you want to do it the same way as myself or not is up to you. Though if you want to you will merely need to run and update something like: UPDATE wp-terms set `name` = `slug`.

Changing WordPress Default Theme Front Page To Show Post Excerpts Only

Posted on July 7, 2009 in php by

This is one of those things that tends to get under my skin. Web sites with long pages showing far too much information on the front page. Realistically I think the best approach is to give users access to plenty of information, but not necessarily having them bombarded with it. For the WordPress Default theme (the one based on Kubrick) this is done by showing only post excerpts as opposed to the entire thing.. You can follow the steps below to this affect:

  1. Open up the index.php page of the theme. You can do this one of two ways.
    1. Manually: locate the folder wp-content/themes/default and open the index.php file
    2. WordPress Admin: look under the Appearance menu and select Themes. Once here select the Main Index Template page from the list on the right hand side of the editor.
  2. On the 20th line (or thereabouts) you will find the text: < ?php the_content(‘Read the rest of this entry »’); ?>
  3. Replace this text with the text <?php the_excerpt(); ?>

That’s it. One note though, I would always comment out any changes I have made so that should something go wrong I can always switch back. If you want to comment it out instead of replacing it then you would use < ?php //the_content(‘Read the rest of this entry »’); ?>

Cleaning Up Ubuntu

Posted on July 3, 2009 in linux by

This is just a quick look at cleaning up an Ubuntu system. This is all documented elsewhere by an abundance of people, some of whom I no doubt took this off in the past and have forgotten where now (If you’re that person, then thanks very much, this stff has been coming in handy for me for a few years now).

Removing Residual Configs

The first step is to remove all of the residual configs lying about your system. These are configs which, as the name suggests, have been left behind by removed programs. To do so the we need to open Synaptic. You can find that in the System menu:

System > Administration > Synaptic Package Manager

Once we are in Synaptic we can click on the status button down the bottom left. This will separate your packages into sections based on their status. The one we are looking for will have the words “residual config” in the name. Select all these packages and right click on them. In the menu that appears choose “Mark For Complete Removal“. You can then click “Apply” at the top of the screen.

That’s it. You have now removed all your residual configs.

Removing Partial Packages

You will need to open a terminal for this one. To do so hold down “Alt+F2“. A box should appear, into which you can type “gnome-terminal” to open a new terminal. In the terminal you simply need to type the following to remove these partial packages:

sudo apt-get autoclean

Once you’ve put in your password etc that’s it. All gone.

Removing Unnecessary Locale Data

Theres a nice little app called localepurge that will get rid of all the locale data that you don’t need. This can amount to quite a substantial amount of space. The first step is to install localepurge. In your terminal type:

sudo apt-get install localepurge

After installing localepurge will require that you configure it with the locale’s you want to keep. It will present you with a list of abbreviations from which you can choose. Scroll down to the one that suits you best (i.e. for myself I kept en_IE.

That’s it. It will run automatically after each install and remove unnecessary locale data.

Removing Orphaned Packages

The app deborphan can grab all your orphaned packages and be used to remove them (be a bit careful with this one as sometimes it can remove something you need – it hasn’t happened to myself but I have heard of it happening). So to install the package, in your terminal type:

sudo apt-get install deborphan

To run the app you just need to type this at your terminal:

deborphan | xargs sudo apt-get -y remove --purge

This will remove all the orphaned packages on your system.

That’s it then, you should have free’d up a good bit of space on your system now and it should be a little tidyer than before!

Renaming Files Using Perl

Posted on June 16, 2009 in perl by

I finally got around to writing a script to rename all my music files to the same convention. That being either <track>-<song>.mp3 or <song>.mp3 with no spaces (I prefer underscores instead) and all lowercase. I know, it looks so simple! But the problem is all those occasions where you have something like <track>_-_<SONG>_-_(<remix by>).mp3 which just looks horrible! What this script does could be done in a lot less lines than I have here, but I thought it might be a good way for someone who is unsure of regular expressions to get a grasp on the topic.

So heres the script:

#!/usr/bin/perl -w
use strict;

# Open 'find' process to list files recursively with paths
open(FIND, "find |");
while(<FIND>) {
 # remove leading / trailing whitespace
 chomp; 

 # Don't rename ourself
 next if $_ eq $0;       

 # create temp file (windows wont allow to rename in place from uppercase to lowercase)
 my $name = $_;
 my $tmp = $_.'~';
 rename($name, $tmp);

 # make lowercase
 $name = lc($name);
 rename($tmp, $name);

 my $newname = $name;
 # remove apostrophes
 $newname =~ s/[']//g;
 # remove round brackets and replace with hyphens
 $newname =~ s/[()]/-/g;
 # remove spaces and replace with underscores
 $newname =~ s/ /_/g;
 # remove where in sequence there is underscore, hyphen, underscore and replace with a hyphen
 $newname =~ s/_-/-/g;
 $newname =~ s/-_/-/g;
 # where there are one or more digits followed by an underscore change the underscore to a hyphen
 $newname =~ s/(d+)_/$1-/g;
 # remove all ampersands and replace them with '_and_'
 $newname =~ s/&/_and_/g;
 # remove underscores where there are 2 or more, and replace with a single underscore
 $newname =~ s/_{2,}/_/g;

 # write out the changes
 rename($name,$newname);

}
close(FIND);

GNS3 – Graphical Network Simulator

Posted on May 19, 2009 in linux, networking by

“GNS3 is a graphical network simulator that allows simulation of complex networks.”

I was recently pointed in the direction of this app as a pretty nice graphical interface to the Dynamips / Dynagen router simulation. It still uses Dynamips / Dynagen underneath, along with Pemu (a Cisco PIX firewall emulator based on Qemu), but allows you to do graphically what you normally have to do using textual config files.

The interface is really nice and you can drag and drop in routers, create links between them, and start and stop them through the GUI. Theres no real difference from what I can see in performance etc, so many people would just turn a blind eye and figure, “Why bother, the config files are handy enough”, but it does mean you don’t have to bother with the configuration for the most part, and can just concentrate on the actual task at hand.

And the best news is it’s packaged for Ubuntu! I’m sure this is the case for other distro’s too but I haven’t checked. So all you need to do is:

sudo apt-get install gns3

You can check the documentation etc out at: http://www.gns3.net/