July 27, 2014

Reading Metadata Out of Nikon NIS Elements Generated TIFFs

Fluorescence image of 
cavitation effects,visualised 
using FITC-Dextran.
If, like me, you use Nikon's NIS Elements to do fluorescence imaging, you would have also noticed that nothing other than NIS Elements can read the metadata that is saved with TIFFs. ImageJ can't read it, the GIMP can't read it, tiffinfo can't read it. This means the only way to recover that metadata, which contains important things like exposure time and pixel size is to open the image in NIS Elements.

This sucks, because NIS Elements isn't exactly easily or widely available, and NIS Elements Viewer doesn't help either - it only opens ND3 files.

Fret not, however. The metadata isn't encoded in any particularly nasty way, and with a little exploration I was able to put together a small python script which will dump out all the relevant information.

Hope this helps someone. It would be nice if Nikon used the standard TIFF tags instead of putting everything in their own tags.


May 20, 2013

Go Away AVN

So it turns out that if you google vaccination on google.com.au, Australian Vaccination Network (AVN) comes up as the second result, right after wikipedia.

Search results from google.com.au for
"vaccination" as of 20th of May 2013
As some of you may know, AVN is an anti-vaccination organisation, part of the anti-vaccination movement, whose core beliefs rests on a retracted paper published by a doctor who was struck off the UK medical register after been found guilty of unprofessional and unethical conduct.

Vaccination is a corner stone of public health policy, and the only means of protection for the most vulnerable amongst us who are unable to receive vaccinations. Through vaccination smallpox was eradicated in 1979, and no child was ever again killed or maimed by it.

A child with smallpox
Due to the efforts of groups like the AVN however, there has been recent decline in the number of children receiving vaccinations against common childhood whooping cough, leading an outbreak of whooping cough in Queensland, and sadly, the death of several infants.

While the Australian Government is taken action against the AVN, the fact it has such a prominent position in Google's search results is undermining the overall effort. Thankfully this is something you and I can do something about, and that is what this blog post is for: part of an effort to increase the PageRank of legitimate sources of vaccination like

If you have a blog, make a post like this one, google vaccination on google.com.au and click on every result that is NOT AVN.


December 16, 2012

Un/check all items in iTunes 11

I subscribe to The Moth, and had previously selected all episodes to be sync'd to my iPhone. Recently I found myself short on space, so I wanted to tell iTunes 11 to stop doing that. Previously, it was a simple case of multiple selection in a list and uncheck, but with iTunes 11, it looked like I was going to have to do it all by hand... for over 200 episodes!

By playing around, I found that if you command-click on a checkbox, it has the effect of un/checking all the checkboxes in the list. This made things more manageable.


October 17, 2012

Unknown control sequence \doublespacing

If you are getting the error
"Unknown control sequence: \doublespacing" when you try to compile your 
beamer presentation in Lyx, change
Document→Settings→Text Layout→Line spacing 
to Single.


September 14, 2012

Notes On Running Calibre On A Server

Calibre is a great tool, especially for converting between ebook formats. Here are some notes for getting it to run on a headless server.
  • The binary installer off the website works fine — ignore warnings about completion and desktop integration failing.
  • The installer will always pollute /usr/bin regardless of the installation directly you choose.
  • You will need the following libraries for mobi conversion:
    • libxi6
    • libxrandr2
    • libxfixes3
    • libxcursor1
  • If you get this message
    • SVG rasterizer unavailable, SVG will not be converted
    • You will need to install xvfb and use xvfb-run, like so:
xvfb-run ebook-convert blah.epub blah.mobi


Nook vs Kindle

Definitely the Kindle. Amazon offers an amazing service, especially their personal document service's email feature. That is simply divine. Furthermore, Amazon isn't nearly so annoying to use as a non-US citizen. I don't have to use a trick credit card or proxies to purchase ebooks. Amazon just sells them to me, easy peasy.

Disclaimer: I have a first generation Nook and a Kindle Touch. My comments above are however entirely based on the services provided, not the devices themselves. So this is probably more appropriate as Barnes and Noble vs Amazon.


August 28, 2012

Thoughts on The Design of SPOT2 GPS

  1. I love the fact the screws holding down the battery cover has little flip up handles so you can turn them with just your fingers, and that they are set so that they don't fall out once loosened. This is top quality design. However, the battery compartment door is not attached to the unit, and it should be, so there is no risk of losing that. Admittedly, it is pretty hard to lose something that is fairly large and orange
  2. Somewhat more importantly, the SOS button should be hooked into the power button, in the sense that if you press and hold the SOS button, it turns the device on and goes into SOS mode. As it is, activating SOS is a 2 step process, requiring you to first turn the device on.


August 23, 2012

Jumpcut is dead. Long live Jumpcut

I used to use Jumpcut a long time ago, and was a big fan of it. At some point, I think around 10.5, it stopped working, and I let it go -- too busy at the time to poke at it.

Recently however my interest in it was piqued again, and downloaded the 0.63 source. To my great delight, it compiles just fine, and once recompiled, works perfectly. Score one for good programming and open source!

If like me you have given up on Jumpcut, it is time to add this nifty utility back to your toolbox.


P.S. If you don't have access to Xcode, or the idea of compiling a program sounds like a bit too much, drop me a line at freespace *at* gmail.com and I will make my copy available.

August 22, 2012

Acorn vs Pixelmator

Pixelmator wins hands down. Compared to Pixelmator Acorn is anaemic in terms of features and costs more than twice as much.

Disclosure:  I originally got Acorn as part of MacHeist, so I didn't pay full price for it. I did pay full price for Pixelmator.


August 18, 2012

Converting Excel 1.0 Files On OS X (or some other OS)

  1. Download SheepShaver
  2. Download "New World PPC ROM" from redundantrobot.com
  3. Extract the zip. This should produce newworld86.rom. Rename this file to ROM and put it in the same directory as SheepShaver.app
  4. At this point SheepShaver.app should run, showing you a folder with a blinking ? inside it
  5. Download "Mac OS 9 Boot Image (120 megs)", again from redundantrobot.com
  6. Unzip the downloaded file. This should produce OS9.img
  7. Access SheepShaver's Preferences, and add OS9.img to Volumes. Hit Save and restart SheepShaver
  8. SheepShaver should now boot into OS9. Ignore the warning about virtual memory
  9. On the desktop, there should be a Unix drive. This is your Mac's system volume. All your files can be accessed through this.
  10. Download Excel for Mac archive from Macintosh Garden
  11. Unzip, this should produce Microsoft Excel folder, inside you will find 5 versions of Excel
  12. Navigate to this folder OS9, and enter Excel 4.0 Folder
  13. Run Excel, and navigate to your Excel 1.0 files.
  14. Open each file, then save it again using Normal Format which is just Excel 3.0. Probably a good idea to add xls to avoid overwriting the original.
  15. Open the converted Excel 3.0 file in modern Excel. Microsoft Excel for Mac 2011 is capable of opening Excel 3.0 files.
At step 14 you might run into a problem I faced: Excel simply won't see any of the Excel 1.0 files! The problem is that Excel is using extended file attributes to determine which files it can open. On less sophisticated systems this is done via file extensions.

To see this, if you copy COMMANDS file out of Excel 1.03's sample directory (you have to convert the image first using Disk Utility), and run xattr on it, you will see this:

$ xattr COMMANDS 

Content of com.apple.FinderInfo is:

$ xattr -px com.apple.FinderInfo COMMANDS
58 4C 50 47 58 43 45 4C 01 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

This is the magic you need. Suppose you had a Excel 1.0 file called foo, you add this extended file attribute as follows:

$ xattr -wx com.apple.FinderInfo "$(xattr -px com.apple.FinderInfo COMMANDS)" foo

After doing this foo should show up in Excel inside ShavedSheep.

If all this sounds like too much for you, send me an email at freespace _at_ gmail.com with subject of "Excel 1.0 Conversion".


An alternative to using Sheep Shaver and OS 9 is to use Mini vMac and System 6. This will let you run Excel 1.0.
  1. Download Mini vMac
  2. Download "Mac Classic ROM" from redundantrobot.com. Extracting the zip produces vmac.rom which is incidentally the exact filename Mini vMac expects
  3. Follow these instructions to get System 6.0.8 running
  4. Drag-n-drop the Excel 1.0 program img file (see above) directly into Mini vMac. You will now have access to Excel 1.0
  5. To get a read/writable disk, create a dmg in Disk Utility. Drag-n-drop this into Mini vMac and you will be asked to initalize and format it. Do so, and then drag-n-drop Excel into this new read-writable disk.
  6. Do your conversion, shutdown Mini vMac
  7. Open the dmg, and it should mount, despite having been initalized and formatted by System 6.0.8 -- backward compatibility for the win
Mini vMac is not as powerful as Sheep Shaver, and System 6.0.8 is as much as it can handle. It is however much smaller and easier to use than Sheep Shaver, and for the purpose of converting Excel 1.0 to a more manageable format is more than sufficient.

Note that you might need to do the xattr trick above.


August 15, 2012

A Layman's Incomplete and Probably Inaccurate Understanding of The GFC

Foreword: It has been four years or so since the world went through the throes of financial upheaval known as the Great Financial Crisis. It was a complex event that will not doubt be analysed in depth by experts for years, if not centuries to come, as an example of what could happen. Sometimes we forget that we are still part of a grand experiment, and that nothing like our civilisation has ever existed before. This means that some lessons must be learnt the hard way. The recent LIBOR scandal however would suggests that the financiers are slow learners...

This is blog posts crystallises my thoughts and understanding of the event, gleaned from various places over the last few years. I make no claims on accuracy or veracity -- it is labelled rant after all.

Lastly, the terms I use below are only loosely connected to their actual meaning in the finance world. For example I use capital just so I can not use the word "money" so much. They mean the same thing here. And money just means something you can pay your debtors with.

Lets begin with some axioms:

  • There is a ceiling to the amount of money a bank can loan out
    • Therefore banks want to... realise the value of their loans as quickly as possible
    • They are therefore willing to sell some loans at a discount, in order to free up money which they can loan to someone else, so they can make more money
  • Perception of low interest rate is absolute, no relative. Thus if banks can borrow money at 1%, they can double the interest on their loans, i.e. 2%, and the perception of low interest rate is maintained. Whereas if the inter-bank loan rate was say, 20%, they can hardly add another percent without significantly affecting the number of people taking out loans.
    • If the profit banks make, derived from the difference between the interest rate at which they borrow money, and the interest rate at which they loan money, is high enough, then it is profitable to take on high risk loans. The increased risk of default is balanced by the greater reward.
Now for the play:
  • Easy credit meant more people are eligible to take out loans, most of which were used to buy houses
    • As a result, housing prices went up, and size of loans increased as a result
    • Some branches are opportunistic, and under-rated the risk of loan applicants so they can give them loans.
  • Banks started packaging high risk loans, which they were giving out due to the high profit margin, with low risk loans, and selling the entire package as low risk.
    • For example, suppose risk is 0..1, where 0 means the loan will definitely be paid back, and 1 means instant default. If one packages nine risk=0.1 loans with a single risk=0.5 loan, then the overall package was being sold with risk=0.14.
    • This practice inflates enormously the value of high risk loans, at the relatively small expense of low risk loans. 
  • These packages were then further manipulated. e.g. taken apart and with individual loans sold at the risk rating of the overall package. This serves to obscure the high risk nature of some of the loans in the package. 
    • For example, company A sells company B the above loans as a single package with risk=0.14. Company B then splits the package into five smaller portions, each with risk=0.14, and sells it to company C. If company C is lucky, it will be getting a good deal: two risk=0.1 loans instead of two risk=0.14 loans. If company C is unlucky, it will end up holding a risk=0.1 and a risk=0.5 loan.
    • These manipulation happened many times, to the point no one can be sure any longer of the true value of any individual loan.
Something happened at this point, I am not entirely clear what, but the effect was that financiers realised that some of the loans they hold are not worth what they are paid, so...
  • They raised interests rates to compensate for the suddenly higher risk of the loans they are holding, and;
  • They tried to sell off those loans who providence cannot be verified.
  • These had the effect of:
    • Increasing the number of defaults, and thus repossessions. As loan holders sold off houses in an attempt to recoup their costs, over-supply of houses depressed the price of houses, bursting the real estate bubble.
      • This meant that loan holders could not recoup their costs, and is thus operating at a loss
    • Devaluing the worth of loans with questionable providence, even if they are in fact low risk, and thus good loans.
      • This meant that loan holders were "stuck" with their loans
  • Both of these combined meant that:
    • Loans were worth a fraction of what they were worth before
    • Loans could not be converted into liquid capital
This freezing of capital, and the devaluing of assets lead to panic in the financial world:
  • Creditors called in their debts, especially from those debtors who specialise in mortages
  • But as mentioned above, the debtor's can't pay: the loans they have purchased with the money they had borrowed could not be converted back into capital.
    • As a result, debtors, often large financial institutions, go bankrupt or require injection of capital, i.e. "bail out"
  • Over all effect: 
    • The collapse of major financial institutions
    • Erosion of trust in the financial systems
    • Devaluation of real estate
    • Increased homelessness as homes are repossessed
    • Increased unemployment as institutions closed
    • Slow down in the flow of credit as creditors become more reluctant to give out loans -- once bitten; twice shy.
    • Extreme devaluation in the assets and worth of investors, from banks to companies 
These effects were not just limited to the U.S. The inter-connected nature of modern financial markets meant that banks and institutions all over the world lost billions if not trillions of dollars in worth, and the reduction in available credit pushed up the interests rates.

More importantly, this was not limited to private companies. Many banks are owned nationally, and thus governments around the world suffer in much the same way as private enterprises:
  • Large part of their worth disappeared
  • Higher interests rates
  • Creditors calling in their debts
Governments also have to contend with:
  • Lower tax revenue
  • Cost of bailouts to maintain social structure


July 22, 2012

Over this weekend I have been working on faerii, a USB controlled RGB LED based around the ATtiny85.

Here is a picture of the breadboard prototype

faerii breadboard prototype
The ATtiny85 is mid centre, and immediately above it is the ISP header. To the left of the ISP header, near the centre, is a 4 pin header that provides USB connectivity. Obscuring it is two filter capacitors, pull resistors, and zener diodes. Finally on the far left edge is a common-anode RGB LED.

Here is a video of it in operation:

The project is based around the hid-data example from vusb. The EEPROM contains control blocks, consisting of 4 bytes. The first 3 bytes specify the red, green and blue intensities, in that order. The last byte specifies a duration. Upon reading a control block, the LED transitions, over the specified duration, from its current colour to the new colour. Currently the duration specified in multiples of 20ms.

Lists of control blocks, which I call a control sequence, can be delimited using delimiter blocks. These blocks have a specified duration value of 0xff. When such a block is encountered, the sequence is restarted  from the beginning. Multiple sequences can be programmed into EEPROM.

The host software performs the job of programming sequences into EEPROM. It also provides additional commands, such as RESET and GOTO. The latter allows a collection of sequences to be stored in EEPROM, and activated individually. This feature was provided to minimise the amount of EEPROM writes required to change patterns. For example, sequences might be stored for an idle and new mail state. Activation then simply requires sending the correct GOTO command, without rewriting the EEPROM which as a limited write/erase cycle.

Currently a total of 62 control blocks can be stored. This is due to incomplete support for writing to arbitrary EEPROM address, and the fact the first block is reserved. It should be no great difficulty to fix this.

Two major features I have no implemented yet are REVERSE and MODIFY-ON-READ. These are animation options, where REVERSE specifies that upon reaching the end of a sequence, instead of starting again from the beginning, control blocks should be read backwards. MODIFY-ON-READ is more interesting. The intention is that after reading and executing a control block, the block's RGB  values should be modified. This allows free-running random colour cycling without explicit programming.

The firmware and commandline utility source code is hosted at github. I will add proper schematics next weekend, though you can probably figure most of it by googling and looking at the photo above.