Software Export of Anthony Thyssen

Anthony's Castle is his home

About Anthony Thyssen

I have made these programs (scripts mostly) publicly available, and you are free to copy, modify and or re-distribute them. Please do not remove my name from any of them. If you make modifications, or have suggestions please mail them back to me so that others may share them.

Encryption

askpass_stars_v1: Read a password while echoing stars (Shell Script)

The oldest version of the ask password while echoing stars. Published here as a FYI. Basiclly a proof of concept script.     I do not expect it to be used.

See my notes in "Password Input", and the specific evolution of this script in "Password Reading using Bash - Ask Pass Stars".

askpass_stars_v2: Use systemd-ask-password if it is available (Shell Script)

A slightly more advance version with fall back to linux systemd command binary command if it is available. This does not have any of the complex password caching options of the next version. So is still quite simple.

askpass_stars: Latest version with password caching (Shell Script)

A shell script to try and read a password while outputting stars so that the user can see that what they are typing is being relieved.

This script has options to let it handle password caching in the Linux Kernel Keyring. (See my notes about "Linux Kernel Keyring", though I have found it better to have calling scripts handle the password caching themselves (see the following programs).

encrypted_function: encrypt a shell function within a script (Shell Script)

A way of storing a small shell script (function) that you do not want to be visible to anyone reading the script. A password is needed to decrypt or run the function.

Any small shell script can be stored, and used as needed, and options are provided to let you store, view, change the function, or change the password of the encrypted function. All other options are passed to the encrypted function when decrypted and executed.

Useful for storing secret password generators. For example to convert a URL into a password for a web site, or let you use a different root password on each machine, based on the machines name.

You can set a TTY_ASKPASS environment variable pointing to a password helper script, like "askpass_stars" above, though it is not required.

Script can also make use of password caching (if enabled) to allow you to re-run the program without needing to re-type the password, at least for a short period of time.

keepout: A shell wrapper around 'OpenSSL enc' (Shell Wrapper)

A wrapper around the 'OpenSSL enc' command to encrypt and decrypt openssl encrypted files. It saves the options that were used as an additional (text) header to the encrypted file. This means the exact requirements needed to decrypt the file is remembered with the file, even if the default options used for file encryption are changed later.

This became especially difficult after openssl v1.1.0 when the password hashing 'digest' default changed (from 'md5' to 'sha256'). Also with the implementation of PBKDF2 password hashing in v1.1.1 (specifically the number of hashing iterations used) the need to save this metadata became crucial.

Basically the ONLY thing "openssl" saves with the encrypted file is some file identification 'magic', and the random 'salt' it generated for that encryption. This is not enough information to correctly decrypt a openssl encrypted file (beyond the password that is). That is all the public metadata associated with the encryption is not included, which means it could be lost if whatever encrypted it changes (as openssl did with its default options).

The "keepout" header saves the openssl options used, and is straight forward and simple to understand, you can easily add or modify options that have changed in the "openssl" command, using a binary savvy editor (like "vim") if needed. For example, you can prepend a header to OLD encryptions, with the appropriate (no longer the default) options needed to decrypt those files, turning them into a 'keepout' encrypted file. You can even do this to plain text files!

If the encrypted files are saved with a ".kpt" suffix (this is not coded in the script), you can use the VIM Autocmd's (see below), to allow you to edit encrypted files directly, with password caching so as to save the file with the same password again.

The script in many ways is similar to what "aespipe" did, though the wrapper (for the "aespipe" program) did not do a complete job, nor was it as future proof, or versatile at "keepout".

encrypt: My File Encrypt/Decrypt Program (Perl Script)

Perl program to encrypt/decrypt pipelined file streams using the perl cryptography module Crypt::CBC to do the task with the derived key and IV parameters, much like the "aespipe" C program.

However this version uses the PBKDFv2 password to derive the encryption key from user passphrase. This is a lot safer than simply using the OpenSSL "enc" option to do a 'Salted' file encryption, which does only a single iteration to derive the encryption key from the user provided password. By using PBKDFv2 to iterated the encryption key derivation, you effectively slow down brute force dictionary attacks making it less practical, without sacrificing the normal usage of the pass-phrase to encryption key hashing.

See the script header for more information.

If the encrypted files are saved with a ".enc" suffix (not hard coded into this script), you can use the VIM Autocmd's (see below), to allow you to edit encrypted files, and making use of password caching to remember the password for saving it again.

Note: This program predates "openssl v1.1.1" which now provides, and recommends, the use of a "-pbkdf2" option. That was the original reason this script was created, effectively obsoleting it. It has basically been superseded by the simpler "keepout" wrapper around the newer 'OpenSSL enc' command.

encrypt.vim: Vim autocmd's to edit encrypted files

These are the "vim auto-commands" I use to let me directly edit files encoded or encrypted in various ways (Gzip, GPG, OpenSSL, and my encrypt and keepout scripts), based on the filename suffix.

My own scripts make use of the kernel password caching to remember the password that was used when a file was decrypted for editing, and re-using the same password to re-encrypt the file again. That way I do not need to repeatedly type it in when saving the file multiple times while editing. Something that has caused me a number of mishaps due to 'finger memory' mistakes, and the resulting loss of the data. The cache is automatically erased after 30 minutes, or at the end of the editing session.

pbkdf2: Perl Equivalent to PBKDF2 algorithm (Perl Script)

A native pure perl implementation of the algorithm, implements the Password Based Key Derivation Function version 2 (PBKDF v2) as per RFC 2898 or PKCS #5. It was found in the obscure Palm::Keyring module on CPAN, from which I extracted and tested for correctness. It can be used either as a loadable module, or directly as a perl program. Of course it is slower than the equivalent C program.

The perl "Crypt::PBKDF2" version, more commonly available, has a HUGE number of dependencies that is simply not required! It causes a dependency hell, that can make it a problem to use on many servers, especially on those you may not have root access.

I verified that it does indeed generate the same results as the OpenSSL PKCS5_PBKDF2_HMAC_SHA1() function. See my notes on the "openssl" command and in Cryptographic Key Derivation.

This was built into the encrypt script above, though this has now been made obsolete by the newer keepout "openssl" wrapper script.

pbkdf2: Access OpenSSL Library PBKDF2 sub-routine (C Program)

A trivial 'C' program to allow command line access to the OpenSSL library function PKCS5_PBKDF2_HMAC_SHA1(). This is the function that implements the Password Based Key Derivation Function version 2 (PBKDF v2) as per RFC 2898 or PKCS #5, but until OpenSSL v1.1.1 was unavailable from the command line.

It was used by older versions of the encrypt script above, but was replaced by the pure perl equivalent to reduce required dependencies, when distributed to other servers.

This and the perl alternative now obsolete, as OpenSSL v1.1.1 has made the pbkdf2 functions available via the command line 'enc' interface. See my notes on the "openssl" command and in Cryptographic Key Derivation.

wordlist_cleanup: Cleanup a password word list (Perl Script)

Small script that takes a sorted dictionary of words and tries to remove words that are plurals, verified, or an adjective of a previous word, or is a common prefix of a longer word. This is to try to make a cleaner wordlist for use in XKCD-like passphrase generation.

passphrase_simple: simple pass-phrase generation methods

Some simple password generation methods of various types (selectable), as discussed in Password Generation

Example output (in order: 4words, uuid, hex, readable):

uneroded~uval~salve~Darra 4848be95-d5c3-4c86-91fc-fd6ccea62a7b 68b329da9893e34099c7d8ad5cb9c940 KUUcAcxruGabgya4fsPtb3zY

Note 'readable' means it does not contain characters that could be misinterpreted, especially when printed. EG no 'Il' chars, or 'O0Q' chars.

passphrase_generator: Generate a passphrase from a dictionary

Using a set of random words generate a passphrase based on the XKCD comic and the "HayStack" principles.

Examples of Generated Passphrases:

Portfolio,Avenge,Kilobyte,Safari ___~DICTUM~Chinese~decoder //frightful|unwitting|veto|omen// crowed.trekking.:::>=-.lezzies /-\-launch-SLIGHT-WISE smuggle_6291_luck_tomato ==[GLIDER][popular][SPRINKLED][thrift]== 0942@Giveaway@Ounce@Telly Certified@Deflected@Enrage@Unfazed:::>=- seorita-challies-^^^-GOODRICH {RHIZOTIC}{maskeg}{kumys}{humet} corrosive=eggshell=8229=oink...... culture:===8%lt;--:poetic:oligarch [statutory][nerd][universal][polymer]~~~ @>-,--|Palatable|Subtype|Agency thrums-[+]-BENCHWORK-curassows Womanlike_:^)_Haunt_Gradation (shorteN)(basebalL)(policY)(populaR):-) __readable~brine~oft~Liberian__

Using '-c' option is provided to generate clean, XKCD or '4words' style, of password.

While an '-o' option forces compatibility with an 'old style' password policies (length, numbers, symbols, capitals), still required by most websites.

I have found a few website that have unreasonably small length limits, making me wonder how they are storing Passwords! One of these, a bank, actually accepts longer passwords, but silently truncates that password beyond a length of 8, unless you go to the bank personally to reset the password to use a more modern scheme!

For more information see Password Generation

Filesystems Mounting and Data Store Encryption

ks: An Encryption Keystore (Perl Script)

KS is a key management system that can handle the binary encryption keys needed for ANY encryption scheme. Though it is currently specifically aimed at EncFS data stores, it can be set up (and has been setup) to handle any type of encrypted data store, that uses command line controls.

The 'key files' are and encrypted file using a user supplied password, (using "keepout", see above) and contain the actual (generally completely randomised) binary master key for encrypted file system, as well as the commands, and configuration data needed for the decryption.

The 'key files' are stored in filenames that look like EncFS files and so can be interleaved into a real directory of a fake EncFS filesystem to further protect the fact you are using such a keystore. The 'key store' (directory where 'key files' are stored) can be physically separated from the actual encrypted data (on USB sticks, or away from network mounts), making it more secure (two factor). For EncFS and CryFS, the configuration file is also stored in the 'key file', so it is not stored with the data. This is one of the major criticisms of using EncFS with a cloud based storage provider. Basically even the 'public' details of the encryption of the stored data is secured. I don't believe in giving a cracker any public help if I can.

Fake 'key files' (name/password pairs) can be added to the 'key store'. These keys can be made to decrypt other data, probably from the same location (interleaved data), or even be made to destroy the access to the real data, to further confuse would be attackers. It means you can give up a password to fake, or less important data, without comprising the real data, creating plausible deniability and prevent rubber hose attacks. Basically a key file could decrypt something else, or run any command!

The 'key files' could instead of holding a master key and configuration data, can be used to hold some other text data. For example passwords for various websites, or your mother's secret sauce recipes.

One example is for the 'key file' data to be a complex executable shell script or even a binary program, that can do other things that you want to keep secret. For example a shell script that holds the password and procedure to access to a ultra secure web site. You then never need to see, remember the details!

Comments welcome.

mount_encrypted: User mount of DM-crypt filesystems (Shell)

With the system "/etc/fstab" setup correctly, you do not need to need to become root to mount a dm_crypt filesystem. This script lets you mount a dm-crypt or luk-crypt encrypted block file systems, with appropriate with password handling for either TTY and X-Windows, as well as good error handling and reporting of any problem encountered.

I previously used this script extensively from command line, shell scripts, and GUI application launchers, menus, and filesystem mount programs, to mount encrypted filesystems given a user password, without needing root or sudo access.

However its use by me has since been superseded by EncFS and the "ks" script above. EncFS allows me to directly back up and/or file synchronize the encrypted data between machines, without requiring anything to be decrypted to do so, unlike a disk encryption methods.

Shell Scripts

Merge BASH History Files (Sourced Shell Script)

Merge the on-disk ".bash_history" with the in-memory shell 'history'. Preserving timestamp ordering, and command order within those timestamps. Optionally removing unique commands (even is multi-line), and/or cleaning our simple and/or sensitive commands, according to defined perl RE's.

Edit in-memory BASH History (Sourced Shell Script)

Edit the in-memory history, so you can look for and remove undesirable history you my have typed on the CLI, either on purpose or accidentally, for example a password or other 'secret' data. Only in-memory hostory is updated, on-disk ".bash_history" is unchanged, until written, or bash saves it on logout.

Bash timestamps are modified so as to include a IEEE format date-time, making them more readable, without effecting BASH reading of the hostory file after editing is complete.

Box_Text: Put Input Text in a Box (Shell Script)

This is a small script that will place the input text in a box. Does not handle tabs.

In some ways this is like the program "boxes" which allows for some very fancy ASCII Art boxing of text, but which does NOT understand even a simple box made from Unicode characters that this script provides!

$ fortune | col -x | box_text -s double -w 76 -i 1 -c ╔══════════════════════════════════════════════════════════════════════════╗ ║ The easiest way to get the root password is to become a system admin. ║ ║ -- Unknown source ║ ╚══════════════════════════════════════════════════════════════════════════╝

Percent: Generate a Percentage Bar (ASCII) (Shell Script)

Generate a ASCII percentage bar, that can be printed to a terminal or included in plain text files. If terminal is xterm then graphics mode characters are output. Output only resolves to using whole characters or a 2% scale.

$ percent_ascii 73 "Test of percentage" Test of percentage 0 10 20 30 40 50 60 70 80 90 100 ├────┼────┼────┼────┼────┼────┼────┼────┼────┼────┤ ▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒

Percent: Generate a Percentage Bar (Unicode) (Perl Script)

A rewrite of the previous percent script to use UTF 'Box Chars', allowing each character to output in 1/8 increments, producing a resolution down to a 1/4% scale. The whole bar is offset right by 1/2 a character compared to previous ASCII version.

$ percent_utf 73 "Test of percentage" Test of percentage 0 10 20 30 40 50 60 70 80 90 100 ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ▕████████████████████████████████████▌ ▏

The above example does not really show the use of color and bold effects in the output.

File_Progress: Watch a process reading a file (Shell Script)

Watch and report the progress of any process that is reading a file. It uses "lsof" to look up the read progress of a command on a specific file, and report how far it has gotten. This script has an option to feed the output into one of the "percent" scripts (see above).

$ ls -Fla -rw-r--r-- 1 root sms 9427379750 Nov 17 23:59 audit.log $ bzip2 -q audit.log & $ file_progress audit.log PID SIZE PCT CMD FILE 21338 9427379750 56% bzip2 audit.log

Homespace: report home disk or quota left (Shell Script)

Look up disk or quota space, used and free, of the users home, and report it in a nice way using either the previous "percent" script, or use a 'zenity' percentage bar.

$ homespace Disk Space in /home ( 26Gb / 50Gb ) or 24Gb left 0 10 20 30 40 50 60 70 80 90 100 ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ╻ ▕▏ ▕██████████████████████████▌ ▏

Timeout: kill long running commands (Shell Script)

A complex script that runs the given command, but will kill it, if the command has not completed in the time specified.

This is useful to prevent network commands taking too long waiting for slow remote servers when the information is not that important. For example when getting a hostname from a network IP, or disk quota when the file system is on a remote NFS server that is down.

The script is completely 'Bourne Shell' based, and uses some very complex scripting tricks to allow it to, exit immediately the command does, without any 'sleep interval' pauses, or leaving behind a long running sleep command. For more details of its development see my notes in "Shell Script Hints, and the section "Command Timeout".

Linux machines often have a C version, also called 'timeout', but that is not always available on non-linux machines, and that is where this program fills the gap.

Countdown: A test command for timeout scripts (Shell Script)

This is just a simple script outputting a countdown reminiscent of a NASA rocket launch. It is used as a test command for testing timeout programs and scripts to see how well they work.

See my notes in "Shell Script Hints, and the section "Command Timeout".

shell_seek.pl: allow shell to execute seek, tell, or truncate, systemm calls (Perl Script)

A perl script that allows a shell to 'seek' (reposition), 'tell' (report) that position, or truncate a file to a specific length using just a open file descriptor. The shell script does not in fact need to know the file path to the file, or if the file even still exists within the file system index (pre-deleted).

For network programming it can also shutdown one side of a network connection. For example to send EOF to server, while still recieving the final result of the send.

The command came out of a stack exchange discussion. Bash read-write file descriptors seek, as a alternative to compiling a C program.

Files and Paths

locate_script: Where is this script located (Shell Script)

Small Script you can add to the beginning of your shell programs to determine the location of the script. This lets you find things like configuration files relative to the scripts location, or read the script itself for things like self-documenting manuals (something I do a lot).

It has worked for me for more than 30 years! And I have used it on Sun3, Sun4, Ultrix, Solaris, Linux, MacOSX, with bourne shells, dash, bash, ksh, and zsh. It should work for any Unix-like environment.

Technically locating a running script has no solution, as it could be a piped into a shell, but in practice it does work. See BASHFAQ (28)

cmdout: Label Command Output (Shell Script)

A wrapper around a command which marks the commands output as being from standard output or standard error. Also reports the actual command given and the final exit status of the command.

A useful information gathering about the program being run. Especially when you plan to later use that command in a shell script, or for co-processing.

Note getting the exit status of a command while also piping its output is generally difficult in older shells. This script was originally a demonstration on how this can be achieved in original bourne shell.

home_backup: Home Directory Backup (Tar file of Scripts)

A push backup scheme that creates snapshot "rsync" backup of your home directory into a "current" sub-directory of the destination (local or remote) given. After a backup (update) is complete, a number of rolling 'cycles' of hardlinked snapshots are created, generating per month, week, day and even hourly snapshots.

A simple "recover" script can be used to list, and restore specific files and directories, from any of the backup 'cycles'.

For more information see Rsync Backups, and Snapshoting

linkdups: Hardlink duplicate files (Perl Script)

Quite a complex perl script that very quickly finds large files that are exact copies of each other. It then hardlinks them together to save disk space.

Files are only size tested initially, with full comparison be performed when a possible match has been found, making this re-hardlinking program very fast. Its complexity is its algorithm for attempting to merge two separate hardlink groups of the same file. Only when all the files of two hardlink groups finally merged together as a single hardlinked group, is disk space saved, so it goes to great effort to find all such files.

The primary purpose of this program is to attempt to re-link files that were moved or renamed in "rsync" backups. This program can thus make incredible disk space savings by restoring the hardlinking between duplicate files. This commonly happens if a directory is renamed, causing the hardlinks in a later rsync backups to not be made, even though the file itself is untouched (just the directory path).

unlinkdups: Break All Hardlinks (Shell Script)

Recursively look at the given files and the given directories and break any hardlink that is found. Basically the reverse of the previous "linkdups" script, in that it will un-merge duplicate files again, while preserving any dates and permissions.

This was needed to remove the hardlinks from files that should not have been hardlinked together. Specifically files in my working home directory that are temporary backups or revisions, configuration files, or SVN copies. This allows the 'separated' files to be able to be edited independently from each other, without a 'vi' or 'cp' modifying ALL the backup copies (revisions).

mv_renum: Rename numbered files (Perl Script)

A simple perl script which will find the largest number in the given filenames, and prepend zeros to the number found in the filenames so they list in the correct numerical order. An option will let you specify the number of digits to make all the numbers instead.

If this script is renamed to be "mv_reseq", it can then be used to re-sequence all the numbers, so as to remove any gaps, or spread out the numbers so as to add gaps to the sequence. This can be useful to insert and re-arrange the numbered order of the files.

I use both forms of the script quite regularly when dealing with numbered files.

mv_perl: Perform complex file renaming (Perl Script)

Rename files based on either a given perl substitution expression, OR using one of the many pre-prepared expressions, based on the scripts own filename.

If the script is linked/copied to the filename "cp_perl" or "ln_perl", then it will copy or symbolically link files to the new filename rather than move or rename them.

Built in perl expressions have been included to rename files to: all lowercase, all uppercase, capitalise words, remove punctuation, replace spaces with underscores, and visa-versa, and many more common file renaming.

These can accessed by linking the script to appropriate "mv_*" names (see internal documentation). For example if the script is linked/copied to the command name "mv_lcase", than that command will rename the given filenames to lowercase.

This script was originally based on a common perl renaming script, the core of which was originally created by Larry Wall, the creator of perl. Many variants exist including "mmv" on many linux machines, and under Debian Linux, "rename".

mv_suffix: Rename files with the wrong suffix (Perl Script)

A simple perl script to use "file" to check the type of a file, and fix the files suffix if it is not correct. Especially fixes uppercase suffixes from Window PCs, as well as set JPEG images to '.jpg' suffix.

I find it amazing how often downloaded files have the wrong suffix.

merge: Rename or Copy files safely (Shell Script)

A replacement for the 'mv' and 'cp' command to use in the previous scripts which adds numbers to the filenames to prevent files overwriting and destroying other files of the same name.

perms: Ensure file permissions are correct (Perl Script)

Read a configuration file and set permissions of the files in my home directory according to that file. What permissions should be set is controlled by a special "perm_list" data file. If the permissions of a file is correct no changes are made (preserving timestamps).

ASIDE: The linux "systemd-tmpfiles" program with its user level "tmpfiles.d" configuration files, can do something similar via its 'z' and 'Z' types (adjust mode, glob, recursive). It does not however make distinctions between data files, executables, and directories.

Text File Filters

multi_seq: generate sequences of complex numbers (Perl Script)

This is like the linux 'seq' command, but can handle multiple sequences of numbers. Each substitution is incremented with the last substitution incrementing before previous ones (unless a '-f' option is given).

Each "printf" number substitution ("%d", "%x", %f etc.) found in the {format_string} is replaced by a number from the comma separated argument ranges, specifying the start,inc,end for that substitution. Sequences can be reversed.

You can have an many "printf" substitutions as you like.

Count up and/or down
Note that the last substitution is incremented before the first
multi_seq "file_%d_%d" 3 3,1

Instead of decimal you can also count in hex (as per perl conventions)
multi_seq "...%02x..." 0x2a,0x4f
not this sequence does include 2e,2f,30,31...

Incrementing ISO standard date (including some illegal dates)...
multi_seq "file_%d-%02d-%02d" 2010,2011 12 31

Generate all posible dates for 2010 and 2011.
The '-f' increments in day-month-year order
multi_seq -f "%02d/%02d/%d" 31 12 2010,2011

randomize: Randomize lines in pipeline (Perl Script)

A simple perl script that can be used as a filter. It basically will randomize the order of all the input lines. Essentially the opposite of the "sort" command.

randomline: Extract one random line (Perl Script)

A perl script similar to the previous one, but only outputs a single randomly picked line from the input list. Sort of a 'pick any one' type filter.

This was designed so it does not need to read in the whole input list into memory, instead only holding the 'current' selection from the list that it has already read. That is it has a very small memory footprint. Of course it will not output the final single random selection until it has finished reading all the input lines, as there is a possibility the last line will be the final selection.

Shell Co-Processing

shell_select.pl: select system call for shell (Perl Script)

A small simple perl script to allow access to the UNIX select() system from a shell script which is handling multiple data pipelines. This is typically required in a complex shell co-process programming technique.

Also see shell_select_example.sh, and shell_select_bc.sh, which are demonstration programs using "shell_select.pl" to handle both stdout for the normal results, and stderr error output from a "bc" co-process. Basically bullet proofing the "bc" regardless of input.

For more information see my own notes on "Co-Processing", and the section on "Co-Processing with both stdout and stderr...".

shell_expect: simple static co-processing script (Shell Script)

A generic but simple co-processing scripting method feeding static requests after looking for specific prompts from the process being feed the data.

Based on a similar script by Steve Parker, "Simple Expect Replacement". See my own notes on running "Co-Processing, in the section on "Timed Data Pipelines".

TTY Capabilities

graphics (Shell Script)

graphics2 (Shell Script)

graphics_colors (Shell Script)

graphics_256color (Shell Script)

graphics_utf (Shell Script)

Various scripts I wrote (starting in 1990's long before the WWW) to check on the ANSI graphics capabilities of my current terminal.

The exact results vary greatly depending on the terminal program (especially for colors and attributes) and on the font you are using. Though they all work perfectly for the "xterm" program. Other terminals tend to be a pale comparision to the original!

Also it seems that many of the special ANSI graphic character modes are no longer functional with the more modern UTF fonts, but then they have other methods to make use the vast UTF characters now available.

$ graphics Graphics Character Reference Screen ──────────────────────────────────────────── Graphic Start {ESC}(0 or ^N Stop {ESC}(B or ^O Never use the later on sun openwin xterms - it stuffs graphics mode! a = ▒ g = ± m = └ s = ⎽ y = ≤ b = ␉ h = ␤ n = ┼ t = ├ z = ≥ c = ␌ i = ␋ o = ⎺ u = ┤ | = ≠ d = ␍ j = ┘ p = ⎻ v = ┴ } = £ e = ␊ k = ┐ q = ─ w = ┬ ~ = · f = ° l = ┌ r = ⎼ x = │ ` = ◆ lqqwqqk ┌──┬──┐ x x x │ │ │ o p q r s tqqnqqu ├──┼──┤ ⎺ ⎻ ─ ⎼ ⎽ x x x │ │ │ mqqvqqj └──┴──┘ * bold* bold` ` bold. . bold~ ~ * * ◆ ◆ . . · · ────────────────────────────────────────────

The above output looks better in ANSI terminal window.

$ graphics_utf graphics UTF-8 Characters 2500 to 2600 _0__1__2__3__4__5__6__7__8__9__a__b__c__d__e__f_ 2500 ─ ━ │ ┃ ┄ ┅ ┆ ┇ ┈ ┉ ┊ ┋ ┌ ┍ ┎ ┏ 2510 ┐ ┑ ┒ ┓ └ ┕ ┖ ┗ ┘ ┙ ┚ ┛ ├ ┝ ┞ ┟ 2520 ┠ ┡ ┢ ┣ ┤ ┥ ┦ ┧ ┨ ┩ ┪ ┫ ┬ ┭ ┮ ┯ 2530 ┰ ┱ ┲ ┳ ┴ ┵ ┶ ┷ ┸ ┹ ┺ ┻ ┼ ┽ ┾ ┿ 2540 ╀ ╁ ╂ ╃ ╄ ╅ ╆ ╇ ╈ ╉ ╊ ╋ ╌ ╍ ╎ ╏ 2550 ═ ║ ╒ ╓ ╔ ╕ ╖ ╗ ╘ ╙ ╚ ╛ ╜ ╝ ╞ ╟ 2560 ╠ ╡ ╢ ╣ ╤ ╥ ╦ ╧ ╨ ╩ ╪ ╫ ╬ ╭ ╮ ╯ 2570 ╰ ╱ ╲ ╳ ╴ ╵ ╶ ╷ ╸ ╹ ╺ ╻ ╼ ╽ ╾ ╿ 2580 ▀ ▁ ▂ ▃ ▄ ▅ ▆ ▇ █ ▉ ▊ ▋ ▌ ▍ ▎ ▏ 2590 ▐ ░ ▒ ▓ ▔ ▕ ▖ ▗ ▘ ▙ ▚ ▛ ▜ ▝ ▞ ▟ 25A0 ■ □ ▢ ▣ ▤ ▥ ▦ ▧ ▨ ▩ ▪ ▫ ▬ ▭ ▮ ▯ 25B0 ▰ ▱ ▲ △ ▴ ▵ ▶ ▷ ▸ ▹ ► ▻ ▼ ▽ ▾ ▿ 25C0 ◀ ◁ ◂ ◃ ◄ ◅ ◆ ◇ ◈ ◉ ◊ ○ ◌ ◍ ◎ ● 25D0 ◐ ◑ ◒ ◓ ◔ ◕ ◖ ◗ ◘ ◙ ◚ ◛ ◜ ◝ ◞ ◟ 25E0 ◠ ◡ ◢ ◣ ◤ ◥ ◦ ◧ ◨ ◩ ◪ ◫ ◬ ◭ ◮ ◯ 25F0 ◰ ◱ ◲ ◳ ◴ ◵ ◶ ◷ ◸ ◹ ◺ ◻ ◼ ◽ ◾ ◿

Also see UTF-8 Demo File.

X Windows

xlogout: Simple button on screen (Shell Script)

Try to generate a simple logout button on the screen (lower-right corner) using any program that can be co-opted into doing this. When button is pressed or killed, the script exits.

To be used as the last command in a xsession or xinitrc script.

I have a similar version in my actual xsession script which on press pops up a menu of options: Poweroff, Reboot, Restart, Logout, Cancel. But the "Restart" option requires integration with the rest of the session script to allow it to kill off and restart all startup applications.

xwin_find: Wait for a window and print its WindowID (Shell Script)

Wait for a specified x-window client window to appear, (or timeout), and return that windows WindowID. This ID can then be used to modify the application window such as resize, move, iconize it. The script is basically a simple looped wrapper around the standard "xwininfo" command.

Currently it could use a re-write at this point in time to make better use of newer x window control tools.

notify_message: Pop-up a small notifaction to user (Shell Script)

Pop-up a temporary notification of some event, letting the user know what is going on in the background, or when some task is complete. It should be as small and minimalistic as posible, not grab the users focus, or otherwise interfere with what the user is currently doing. They should timeout and pop-down (exit) on their own, without user intervention.

A great deal of effort is put into ensuring multiple notifications do not popup on top of each other, but remain neatly ordered in the top-left corner of the screen.

jiggle_window: Window shake or bounce (Shell Script)

Jiggle (move around) a window in some specific way so as to highlight the window or some condition to the user. Window is returned to it starting location when the action is complete.

By default it will cause the window specified to 'bounce' like a ball to highlight its existence. Other actions include 'shake' left and right, which is commonly used to indicate some error condition (bad password). Or do cirlces or jump back and forth.

There is both a xwit version and a xdotool version available. The scripts are identical, just using different window control tools.

capture_ocr: grab and convert on screen text to clipboard (Shell Script)

Look for and convert on screen text to actual text in both a clipboard and a pop up window, using OCR (Optical Character Recognition).

This program should be linked to some 'key event', such as 'Meta-Print' so you can make a image selection (to find text in) at any time.

For methods of doing this see X Window Event Handling.

edit_textbuf: Edit a Web text buffer in VI (Shell Script)

Grab the text from current text buffer input field, pop up your preferred editor with that text. When Editor ends, paste the text back into the original text buffer.

This program should be linked to some 'key event', such as typically provided by a window manager. For example can it when user presses a 'Meta-E' key. See X Window Event Handling.

keyboard_macro: (Shell Script)

Type a string or STDIN, as if you typed it yourself from the keyboard.

This lets you setup special 'hot' keys that can type fixed strings (like an email address) or general text selections into ANY input box, whether it be a web browser input form, or a game input window, regardless of if it accepts a normal 'paste' or not. The application will see the string as coming from the keyboard, and not as a 'paste' which some applications do not accept. Very useful.

See X Window Event Handling.

xmonitor: layout X window monitors in common ways (via xrandr) (Shell Script)

Read what video monitors are available to the display and lay them out in common configurations. Script can be used in session startup scripts, or from window manager menus. It is basically a wrapper for the more complex "xrandr" command, but far simplier to use.

List the monitors simply:	xmonitor list
Clone display to all monitors:	xmonitor clone
Swap to next active monitor:	xmonitor swap
Enable secondary monitor only:	xmonitor second
Left to right order:	xmonitor right
Left to right order (skip first):	xmonitor -skip right

WARNING: If a monitor is not working properly, you could be left without any working display. Caution is recommended on "swap" and "second" actions. Linking a "xmonitor clone" command to a hotkey is a very good idea, just in case.

System Administration

logrotate_one: logrotate using one sub-conf file (Bash script)

Run the logrotate against a single sub-configuration file, while also defining the global settings from /etc/logrotate.conf

Graphics

thumbs_cleanup (perl)

A perl script to recursivally look for and cleanup thumbnail sub-directories associated with XV, and GQView. It can forcefully remove all thumbnail sub-directories.

Gif trans (.c) and Man Page (.man) and Text Manual (.txt)

This is a patched version of the "giftrans" program. That fixes some reported GIF file setting numbers (bad rolls) and comment handling of the more modern "rgb.txt" files. It should be compiled with a RGBTXT define giving the location of the "rgb.txt" file, though it is not strictly necessary for its correct working.

I use this program in some of my icon library scripts to gather information about GIF files, such as the exact colormap, disposal, and delay settings from GIF animations.

Also see ImageMagick Examples, Helper Scripts, and especially the "gif2anim" script. This script originally used this program for information gathering to create a ImageMagick command that can re-build the animation from its de-composed frames. However it now uses ImageMagick itself to gather this data.

Minecraft

journeymap_join

Using Imagemagick join the tile images generated by journeymap mod, of places the player visited on Minecraft servers, into a large map.

Note that images can get very large for some servers, so you may want to specify a center point and limit (in terms of 512 block tiles) that you want to join together. This can also be used to limit the area to just around your base.

You can specify the map set, (dimension), and if you want day, night, or specific underground level maps.

journeymap_dzi

Create a Deepzoom (DZI) pyramid image that can be displayed on a web server to explore very large (and sparsely mapped) minecraft server maps of places you have expored on that server (via Open Seadragon java script). The deepzoom image lets you see the whole zoomed-out world, showing the areas of the world that have been explored, and then zoom-in to pixel level in any of those areas, and pan around with complete freedom. Extra code converts clicks on the map to minecraft coordinates, while double click zooms in one more level.

The DZI image format handles sparse tiling efficiently (no need to include empty tiles in the image). The script can 'symlink', or 'hardlink', the original journeymap images for the DZI deepest level with the appropriate image filenames, allowing them to be used directly by the DZI without duplicating ('copy') them. This saves more that 1/2 the disk space needed to hold the complete DZI image. From these original journeymap tile images other higher (zoomed out) layers of DZI pyramid image, are created.

The script is also designed so you can just run it again (with same options, if any) to add and update new journeymap tiles, and the the zoomed out images, as needed.

FUTURE: I can generate markers on the DZI image for journeymap 'waypoints' However this is not simple, with too many 'server specific' things to allow me to publish that script. Mostly it is reading and converting the Journeymap waypoints into a JSON file that the DZI image javascript can use to create the markers on the DZI map.

For more information see notes in Journeymap to Deepzoom Notes, and look at the online example of the results (with waypoint markers) at Core Journey Deepzoom Map