Tom O'Hara Shell Script Collection

This directory contains a distribution of my collection of useful shell script utilities, predominantly in Perl. Most of these have been developed at NMSU, such as in support of my thesis work (O'Hara 2005). Also see Thesis code. Several of the scripts were developed in support of the GraphLing project directed by Janyce Wiebe and Rebecca Bruce. However, some were developed or refined at Cycorp (and thus are partially copyrighted).

Links to individual scripts are shown below, along with a brief description. In addition, a tar archive of the collection is available:
     tpo-useful-scripts.tar.gz.

The scripts are free in the GNU sense (see license below). However, if you find the scripts useful, please make a small donation to the NMSU Computer Science Department (or not so small, if really really useful).

Tom O'Hara
Summer 2006

Thesis info

O'Hara (2005), Empirical Acquisition of Conceptual Distinctions via Dictionary Definitions, PhD Dissertation, New Mexico State University, 2005.

Disclaimer

Copyright (c) 2005 Tom O'Hara and New Mexico State University


This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.


This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.


You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

anova.lisp: xlispstat source for running ANOVA analysis over two sets of data

ascii2network.perl: create a belief network from an ascii representation

belief_network.perl: Perl module for producing belief networks in the formats required for BELIEF and MSBN (Microsoft Belief Network).

bib2text.perl: convert bibtex bibliography into free-text format

bibitem2bib.perl: convert from bibitem entries into bibtex format

calc_cooccurrence.perl: Calculate co-occurrence statistics for the input, each line of which represents a 2x2 contingency table.

calc_entropy.perl: calculates entropy of a class using the output of a frequency tabulation program (in particular count_it.perl).

calc_multi_x2.perl: Calculate chi-square & G2 statistics for each line of the input, each representing a 2x2 contingency table.

calc_x2.perl: Calculates a simple X^2 (chi-square) independence test for a 2x2 contingency table

calculate_weights.perl: Calculate the weights for the salient words in the context of a Roget's category.

check_errors.perl: Scan the script execution trace log for errors, warnings and other suspicious results.

cmd.sh: run's the specified command, capturing stderr

common.perl: Commonly used perl routines (variables & constants)

convert-todo-to-outlook.perl: convert todo list from from todo file into a task comma-separate value (CSV) file for import into Outlook

cooccurrence.perl: perl library routines for word cooccurrence statistics.

count_bigrams.perl: count bigrams (and n-grams) in a file

count_it.perl: script to count the occurrences of a pattern in the input

cut.perl: cut columns like the cut utility but w/o line restrictions

derive_contexts.perl: Derive corpus-based contexts for each of the categories from Roget's Thesaurus

determine_unix_shell.perl: determines which interactive shell the current process is running under

disambig_entries.perl: disambiguate translation entries

disambiguate_text.perl: Disambiguate the words in a text by finding the Roget's category that best matches the text's context and then selecting the sense that applies in the context of a Roget's category.

do_regression.perl: perl multiple linear regression on the data specified in the input

do_tex.sh: front-end script for running TeX optionally w/ BibTex

dobackup.sh: Make a copy of the specified files (in the current directory) into the ./backup subdirectory

domainname.sh: wrapper around domainname in case under CygWin

example-test-support.perl: support code for test-perl-examples.perl

extra.perl: common module with extraneous but occasionally useful functions

extract_factotum_relations.perl: extracts the relations from the Factotum knowledge base (by Micra)

extract_lex_rels.perl: given a part-of-seech tagged file of dictionary definitions extract the lexical relations implied, using pattern matching (eg., 'GenusNP PASTPART ..

extract_matches.perl: script for extracting text in a file matching a particular pattern

extract_roget_categories.perl: Extract the Roget categories from the public-domain 1911 version (prepared by MICRA)

extract_subfile.perl: Extract a portion of a file, base on inclusion regions.

extract_synset_freq.perl: Extract the frequency for the WordNet synsets from the semantic concordance (SemCor) sense counts (cntlist).

feature2ml.perl: convert from tab-delimited feature specification to format used in the ML repository (c4.5-based)

filter_biblio.perl: filters out a subset from a bibliography

filter_mail.perl: Ad hoc script to extract mail messages from a series of mail files

foreach.perl: Evaluate a command for each argument

foreach.sh: Evaluate a command for each argument

format-weka-decision-tree.perl: convert Weka's decision tree output into latex format

generic_client.perl: client for accessing the MSBN inteface (DSC) remotely

generic_protocol.perl: Module for implementing a client/server protocol

generic_server.perl: server for transferring files and running commands

intersection.perl: show the word overlap (or difference) of two files.

join_files.perl: make each file a separate column of the result

k_vec.perl: program to compute k-vectors for use in parallel corpus alignment.

kill_em.sh: script for killing processes specified by name or pattern

l_vec.perl: program to compute l-vectors for use in parallel corpus alignment.

misc_xlispstat.lisp: Miscellaneous supporting routines for using xlispstat for basic statistical analyses.

normalize.perl: normalize the vector given on the command line

paste.perl: Utility similar to Unix paste command for joining columns

perlcalc.perl: simple calculator in perl

perlgrep.perl: quick n' dirty version of (e)grep via perl regex matching

prep_brill.perl: preprocess a file prior to part-of-speech tagging via Brill Tagger

process_network.perl: Process a belief network from scratch

produce-script-index.perl: produce an index of scripts suitable for incorporation into a HTML directory listing

produce_call_graph.perl: Produce call graph for specified scripts

ps_mine.sh: show processes belonging to particular user note: the processes are shown sorted by CPU and then by memory

ps_sort.perl: produces top-like Unix process data listing

qd_analyze.perl: quick & dirty analysis of Spanish text

qd_eval.perl: quick & dirty evaluation of align_word.perl results

qd_trans_latin.perl: quick and dirty Latin translation of text

qd_trans_spanish.perl: quick and dirty translation of Spanish text

random.perl: Outputs a list of random numbers.

randomize-lines.perl: randomize the contents of a file line-by-line

rcsput.perl: wrapper around rcs ci that handles locking

remove_carriage_returns.perl: remove carriage return characters from sources files (e.g., \r\n => \n)

rename_files.perl: script for renaming a list of files by doing a simple pattern replacement (old to new), assuming Unix (or that the DOS version of the mv command is available).

reverse.perl: reverses the lines in a file

roget.perl: misc script for working the Roget categories

round.perl: rounds all numbers in the text (in place)

rup.sh: simulated rup using uptime

server.perl: Simple TCP server (for testing purposes).

set_xterm_title.bash: Sets the xterm title bar to the string given on the command line, including support for CygWin (via cmd /c title).

shortest_paths.perl: Compute the shortest paths between all pairs of nodes in a graph.

show_directory_tree.perl: recursively traverses the directory structure, creating GraphViz graph displaying the subdirectory names, including support for symbolic links.

spanish.perl: module for processing Spanish text

spanish_lookup.perl: Looks up the word in the Collins Spanish-English dictionary

split_subfiles.perl: Split a portion of a file, base on inclusion regions.

strict-common.perl: version of common.perl that specifies 'use strict'

sum_file.perl: script to sum the first column of numbers in a file with support for optional statistical analysis via xlispstat

synch_directories.perl: synchronizes the files in two directories roughly in the manner of the Windows-based briefcase facility

test-perl-examples.perl: Script for performing unit tests on functions in perl scripts by checking for examples flagged with 'EX:' in the function header.

testwn.perl: script for testing the WordNet access module

trans_spanish.perl: quick and dirty translation of Spanish text

web_freq.perl: check frequency of phrase on the web via search engine queries, currently Hotbot or Altavista

win32.perl: module for interfacing w/ the Win32 module

wn_distance.perl: calculates the "distance" in WordNet between the words

wn_polysemy.perl: retrieves the WordNet polysemy for a list of words

wordnet.perl: module for accessing the WordNet knowledge base (i.e., WordNet access functions)

xlispstat.sh: invokes the right version of xlispstat for the platform (Linux or Sun Solaris), using the required resource/working storage file (xlisp.wks)