DaveSource Marginal Hacks

"I'm glad cavepeople didn't invent television, because they would have just sat around and watched talk shows all day instead of creating tools." - Dave James

LICENSE Is it Free?

Too see more/less tools, change minimum rating (0,3,4,5 stars): none |
-- Currently displaying code with 3 or more stars --

Name Rating Works? Description

airtunes

Simple perl script for UNIX/Ubuntu/Debian systems to stream mp3/oggs to an AirPort Express Airtunes server. Requires Justeport (which requires mono to run the .NET exe)

album

An html photo album generator which descends directories and has theme support. For an example, see my photo album.

AmazonCovers

Gets CD cover art from Amazon.com for an entire tree of 'Artist/Album/' directories. Mostly works for me, has trouble finding some artists. Your mileage may vary. (I use this with XMMS and the groovy CDcover plugin and love it..) Requires Net::Amazon CPAN module as well as lynx or GET.

asq

Mame Cabinet front end, nice and simple and fast, using arcade controls. It runs in a terminal window but can display a screenshot window as well. Supports MAME as well as other emulators and various games.

bew

A webcrawler/robot for mirroring entire web sites or web trees. Just give it a URL and it gets everything underneath it. It can tell you about broken links as a helpful side-effect. If you want something more stable but don't need the recursion, see findex.

BMP.pm

A perl module (Image::BMP) that can read, modify, save (and even ASCII display) BMP bitmap format images. Use 'perldoc BMP.pm' to read the docs.

bugz

Bugzilla scraper. Can get list of bugs from a bugzilla server and output them in a user-definable format. Useful for regression scripts and the like, or anyone who wants command-line read access to bugzilla.

burstAnim

Converts multi-burst photos taken on a digital camera (such as my Sony DSC-L1) into animated gifs.

colmath

A simple script that cuts columns from files (like 'cut') and allows you to do 'math' on the results, such as adding the fields from one file and removing them from another. (This is intersection/combination type math, not number math - this allows, for example, for you to list all the email addresses in one file that don't appear in a second file).

Contact.cgi

A CGI mailing form that can verify from email addresses without revealing your email, as I do at my Contact Page. Great for making a "Contact Me" page without worrying about people collecting your email. Also good for mailing lists and other websites that want to replace all their mailto: links. Themeable.

Mail.cgi

A simpler CGI mailing form that doesn't verify email addresses but can only send to a hardcoded destination. Themeable.

dateCalc

Does math with dates and times. As examples: "dateCalc today-6d3h2s" "dateCalc 1/2/2010-12/3/1971", etc..

daveboard.tar.gz

Simple CGI bulletin board. No login required, but no editing/deletion of messages and no threading. Good for a simple board where users are essentially trusted. Read top of file for install.

Davent

Themeable event registration system which I've developed/used for our dance weekend events for over a decade now. Includes website creation, registration, payment processing, door checkin and even post-event survey. Different events are created by creating small perl files of info, allowing things like complex date-based or # sold based pricing. The biggest weakness is in dealing with more complex checkin issues, such as dealing with one email buying multiple registrations and doing registration changes and trying to figure out the correct price (some of these problems are intractable).

DaVite

This is a web invitation system, much like evite.com or invites.yahoo.com, but you have total control over it! Note: I am no longer developing DaVite, as I have started to do work on MyVite

debed

Finally, sorenson plays on linux, but unfortunately, it can't handle those streaming embedded quicktime movies. debed can find the actual movie file for you, thanks to info posted on slashdot Sometimes -fetch might not work depending on the setting of $GET and whether the website you fetch from allows it.

define

Looks up word definitions, thanks to the webster gateway

ePerl

Embed perl in your text files! This is a very snifty tool. It was originally written by Ralf S. Engelschall as a C program, and I realized that it should be written in perl, which I did in a program that is an order of magnitude smaller. (In fact, the source for this page is eperl).

every_change

This script kicks ass for code development. Watches a file and runs it (or something else) every time it changes. Write your code in one window, and watch it automatically execute in another.

every_line

A little less exciting, this runs a command on every line of input it gets. I've thought about adding the ability to pick fields from the input, but..

exec_trace.tar.gz

An LD_PRELOAD tool that can give you a log of all the exec's done by a command and any of it's children. Useful for debugging wrappers that call wrappers that call wrappers... Also traces stdout/stderr file redirects as well as environment variable settings.

File_Browser.cgi

This is my file/database browser in a cgi script. It has a similar look as the old Windows file browser except that it can keep track of open folders inside closed folders. You can see it in action at the Fringe.

find_dup

Efficiently finds all duplicate files in a file list and optionally removes them. Now (v2.00) with massive speedups for large sets of large files. Ran on 16Gb of 3900 files: old find_dup: 12 minutes, new find_dup: 3 minutes.

findex

Fetches an index of files. Can count through numbered URLs and only grab specific types of files, etc.. Pretty darn useful for scrubbing links out of pages. Like a non-recursive but more stable version of bew. See options/usage for many ways to grab many URLs.

fix_img

Run ImageMagick 'convert' on a batch of files, this can sometimes repair broken images. Can also be setup to easily run common convert arguments such as geometry, normalize, etc.. Simple and very useful for image processing.

flashcache

Finds/copies the hidden/deleted flash cache files that chrome (and firefox?) now store with the flash plugin, such as youtube and google videos (can find other such deleted cache files as well). UPDATE 2012/05: Seems that youtube has changed their ways and this no longer works, but I've updated myfilm which should be working again. If you have any information on how Youtube is caching their files, I'd love to hear it.

former

Creates CGI form/registration code. You describe the form inputs, and it creates the code for outputting, checking and correcting. It's pretty damn useful if you ever need to write some forms. You can look at the script directory, or else grab the tar.gz

forge

Forge email. Can handle port 25 and sendmail method. If you use it by itself and don't have access to an insecure port 25, then your mail may have the "may be forged" messages. You can fix this by either downloading the suid version or, much safer, downloading the additional forge_pipe tool which forge can use for you and then:
chown root:root forge_pipe ; chmod 4755 forge_pipe
I don't guarantee that it's safe as suid!

google-cal

A command-line interface to google calendar using ruby. It can also add Facebook events, extracting the event code from the Facebook page. A good starting point or example code if you want to write your own interface to google calendar in ruby. Includes a setup wizard for handling the authorization process as well as script installation.

hex

Simple and kicks ass. Type in decimal, octal, hex or binary, and this outputs all the possible formats.

html2jpg

Converts HTML pages to images (gif, jpg, png, ps, etc..). Very kludgy, only works with certain versions of Mozilla, but I needed a way to automate screen grabbing the HTML window out of a browser (for my album project). Maybe it'll help you. If you have KDE, I recommend khtml2png

httpdave

Piece 'o cake webserver. No config files, no setup, no compilation. Just run it in your web root directory and go to http://localhost:3001/. Great for quick testing to see how HTML/CGI will look (I use it alot for album development).
Handles CGI, server-side-includes (.shtml!), cookies, mime types and POST.
Older versions of perl had two socket bugs that would 1) cause crashes when a download was canceled and 2) cause infinite loops when trying to send a 301 message if a '/' was missing at the end of URLs

img2iphone

Transfer images to your iPhone so the show up in "Photos" and the "Camera" applications. Can use iPod-convenience package to do the copy, otherwise can create a local directory of iPhone images that can be transferred manually with scp. Requires ImageMagick (convert/identify). Can only transfer to "Camera Roll" - if anyone knows how to create other albums, please let me know. Updated 2009/12/26 to work with v3.0+ firmware. Also see vid2iphone (doesn't do transfers).

LocationExample

Example Android project that creates a simple but complete location app. It uses Google Play Services and Google Fused Location Services (the latest location API) to do location updates and properly handles the lifecycle as well as turning location lookups on and off.

lnR

Recursively links a bunch of directories with either soft (-s) or hard links. It makes a matching directory structure and then links all the files in each one. You can also specify a regexp to -ignore certain files. Useful for copying a large file tree for testing without using alot of diskspace. Can also keep a 'history' of files seen so you can remove them from the new tree and not have them re-added if you rerun lnR.

makeKML.pm

A perl library that writes KML files for Google Maps and Google Earth. Can do the Geocoding (latitude/longitude lookups by address) using the Google API or OpenCageData.com. Includes a simple HTML example that embeds the KML map using the new Google Maps Javascript API v3 (because Google Maps no longer supports direct usage of the KML on maps.google.com). See "perldoc makeKML.pm" for information.

make_faq

Makes FAQs and such things. I use this to build: DaveFAQ and DaveHistory

map

Calls up your browser showing a map given an address in various formats. I'm pretty sure the URL it builds is wrong, but that's easy to update. You need to have a browser that can pop open a new window when given a URL as an argument. You can do this with old versions of netscape with a wrapper and -remote, which kicks ass.

mzmv

Moves mp3/ogg file(s) according to their tags. Replaces the 'mp3mv' tool I had here previously. Requires vorbiscomment for oggs and/or id3tool for mp3s

mpls

Make extended xmms playlist. Takes a list or directory tree of mp3 and ogg files and creates a playlist sorted by artist, album, track, title. More importantly it can write EXT metadata for xmms (Turn on "Read meta data from playlists" in preferences) so your playlists have song name and duration info immediately available to xmms. Is currently butt-slow for oggs because ogginfo is butt-slow, run it as a cron job.

mvx

Simple yet kicks ass. Moves or copies files using regular expressions. Thanks to Andrew Allen for the idea, which I'm sure he stole somewhere else.

myfilm

Downloads films from youtube and iFilm to your hard drive. Converts youtube flash films to .mov if "ffmpeg" is available. If the format changes, let me know. Works with youtube as of 2011/12. Works with iFilm as of 2006/05. Requires perl and "lynx" or "GET"

oggmv

Moves ogg file(s) according to their ogginfo to a proper hierarchy (like id3mv)

ogg2mp3

Convert oggs to mp3s, converting ogginfo tags to id3 tags. Requires: ogg123, ogginfo, id3tool and an mp3 encoder, perl and nothing else. I've had a bug report that this might be limited to 48 minutes, I'd love more info on this.

perlc

Perl to C 'compiler' for hiding perl source code and creating an executable for distribution. Also has simple support for limiting distribution. (Not true compilation!)

perlc.zip

A variation of my perlc actually written in C by Dimitar D. Mitov with a number of features: Can automatically detect libperl.a, does dependency scanning, static linking, AES 256 CTR encryption using disk device or 1st Mac address. (Not true compilation!)

pemf

I got tired of procmail and the like, so I just wrote my own mail filter that's easy to control and can do simple things like remove MIME. MIME sucks. If you can't vi it, it sucks. It can also catch SPAM, auto respond to email, forward email, blah blah blah. Everything that procmail can do, but it's perl, so it's better. Perl rocks.

psh

Perl shell, almost full featured. Features: Completion, history, aliases, environment vars, dynamic tcsh-like prompts, source, globbing, job control,..

Random
Signature
Generator

Consists of: sig.pm, mail_editor, diffsig
Random email signature generator. Instead of using pipes or cron job kludges, this simply puts a wrapper around the mail editor that my mail program invokes. This chooses the new sig, and if you don't like it, you can use diffsig to change it. I use vim, which kicks ass, so I have this mapping so I can change my sig with ",s":
.vimrc: map ,s o<ESC>:.,$d<NL>Idiffsig <ESC>!!sh<NL>
I didn't write this for other people, so you'll have to read through the code to figure out where to put all the files, sigs and whatnot.

rdiff

Remote diff (with recursion). Similar to rsync or scp, but shows diffs between a local directory/files and the same path on a remote host. Currently requires that the absolute paths are the same on both machines, but it wouldn't be too difficult to make that an option.

scurvy

Lets you easily write screenplays or scripts in a simple text format, then converts them into the proper screenplay format. Also imports some RTF and Final Draft formats and can convert to multiple formats as well.

serverizor

Converts a generic command into a tcp server so you can disconnect and reconnect to it from the command line. This is actually pretty handy.

site_index

Multi-page HTML site index generator for local domains. See examples at any of my domains. Companion to whats_new

spigot

Spigot is an easy way to turn on and off printing of text according to patterns matched and other simple rules. Like a combination of 'expect' and 'cat' and 'head'. See usage for simple examples.

splitCollageAlpha

Simple ruby script (using imageMagick rmagick gem) to split up a collage of rectangular images separated by transparent boundaries. If your boundaries aren't transparent (yet), take them to an image editor (like Gimp), select by color, make transparent (after adding a transparent layer if your image doesn't have one) and then save to an image format that supports transparencies (such as PNG). The rectangular images do not have to be on a standard grid, every column/row can be different sizes (in terms of number and pixels).

sqlite3_hash.gem

My first published ruby gem. A persistent simple Hash backed by sqlite3. Contains (almost) the same features/API as the Ruby 2.0.0 Hash object. Also completely tested with rspec, making it a reasonably simple rspec tdd ruby example. Also on github

tcp_forward

Forwards a TCP connection and lets you snoop and filter the traffic being sent. Uses the simple IO::Socket server/client examples in the perlipc manpage.

thumb

Just a thumbnail generator, except that it makes thumbnails the same size while keeping aspect ratio by cropping - so the pictures aren't stretched. This looks better than I expected - and is the code I use for the cropping mode of 'album'

upchanged

Recursively uploads all files that have changed in a directory structure since the last call to upchanged (using a touch file, tar and ssh)

vidb

Simple BerkeleyDB database text editor (Probably smarter than using DB_File) (As always, If you can't vi it, it sucks)

vig

vi-grep, or emacsg (emacs-grep), pineg, etc.. etc.. This will search files (or a hierarchy of files) for regexps and bring them up in an editor or list them. Very useful for managing source code if you are IDE-allergic, as I am. (As always, If you can't vi it, it sucks)

whats_new

What's new generator for local domains. See my What's New page. Companion to site_index

xags

A complete black-box copy of GNU xargs v4.1, written in perl. Why? Just for kicks, and so I could fix some bugs and add an option. I also have a man page (html) for it. This has been submitted to the PPT Note that the code is not derived from the GNU source. Also, I have some snifty simple testbench code that helped me make sure it matched xargs in every way.

xnewmail

Simple yet kicks ass. I use ssh and elm/mutt for my mail. This allows me to have a background job that watches my remote ssh server for new mail and then launches a mail window when needed. I had a previous non-remote version that I used display forwarding with called xnm

Gets yellow page listings on the command line, courtesy Yahoo
Works as of 7/4/2000 [updated for 10/28/2002], and will continue to work as long as the format at yp.yahoo.com doesn't change drastically.

vid2iphone

Converts videos/movies/DVDs to a movie format suitable to the iPhone. Requires ffmpeg (with libfaac support!), possibly mplayer, and a UNIX-y system. Most Ubuntu ships ffmpeg *w/out* libfaac support, you will need to recompile (check google). Also see img2iphone.

qPoll

This is a CGI polling program. I didn't write this code! I briefly worked with it when I fixed a security bug in it, and the home page and author have since disappeared, so I have a local copy available here. I am not the maintainer of this code! It is just here for convenience.

xstroke

Very old but very useful in some situations. This allows you to draw "gestures" with your mouse button that can be interpreted as commands. These days this is fairly common, but I wrote it a million years ago when it wasn't.

libExpr.rb

Ruby expression tree generator. Can convert expression strings such as "(7*3)+(var/2)" into an expression object that can be very quickly evaluated (and can handle offloading variable evaluation). Can convert to RPN, Prefix and AST (abstract syntax tree) and has a flexible precedence model that includes presets for Ruby precedence or C precedence. Uses the shunting yard algorithm, but also supports unary (such as '-a' or '+a') as well as ternary (such as 'a?b:c') operations as well as short-circuiting logic operations (such as 'a||b'). Only AST supports all of these features, see the docs at the top of the source for more info). Also has a testbench that generates random expressions and verifies correct operation. It's a tar.gz, but you can also just look at the source and the testbench here.

libMarshalable.rb

Ruby "Marshalable" class with simple exclusion mechanism for Marshal.dump. Inherit this and use 'marshal_exclude' (much like 'attr_accessor') to specify any variables that should be excluded from marshalling, so you can avoid TypeError exceptions.

Author: David Ljung Madison Stellar [CONTACT]


LICENSE	Free?	Reporting Bugs

"I have an elegant script for this, but it was too big to fit in the margins.."

- Me, MarginalHacks

Thanks to Jamie Zawinsky and the Jargon File for the name

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult. - C.A.R. Hoare