2022-12-04

FSCK'ing Veracrypt Volumes on macOS

Due to unexpected power loss or disconnection of the underlying storage, the filesystem inside Veracrypt volumes is desired to prevent future data loss.

On macOS I tried using 

diskutil verifyDisk /dev/diskXXX

This didn't  work for some reason, most likely because of some interaction with veracrypt. Same results via Disk Utility application.

The Veracrypt GUI however offers a "check filesystem" option if you right click on a mounted volume. Unfortunately, as of version 1.25.9, it doesn't work out-of-the-box on macOS: it fails to find 

/Applications/Utilities/Disk Utility.app

The cause is that from at least macOS 12, Disk Utility is located at

/System/Applications/Utilities/Disk\ Utility.app

There is a quick and dirty workaround

cd /Applications/Utilities
sudo ln -s /System/Applications/Utilities/Disk\ Utility.app ./

With the above symlink in place it is possible to perform filesystem verification and repair via the context menu items offered in the Veracrypt GUI.

2022-12-03

Verifying Local iOS Backups

Backups are a great idea, and encrypted backups are a even better idea. When backing up iOS devices on macOS, it is possible to set an encryption key to protect local files. There is however no straightforward GUI method to verify the encryption key.

Enter mvt:

conda create -n mvt python=3.10
pip install mvt
mvt-ios extract-key /path/to/backup

With the right password you will see

INFO [mvt.ios.decrypt] Derived decryption key for backup at path /path/to/backup is: "12345678901234567890"

Otherwise

CRITICAL [mvt.ios.decrypt] Failed to decrypt backup. Did you provide the correct password? Did you point to the right backup path?

2022-07-30

TigerVNC and Systemd

A quick systemd service definition file for tigervnc on Ubuntu 20.04

[Unit]
Description=tigervnc

[Service]
Type=forking
ExecStart=/usr/bin/vncserver -localhost no -depth 24 -geometry 1920x1080 -noreset
User=steve

[Install]
WantedBy=multi-user.target

Key components:

  1. Type=forking so systemd correctly identifies the real VNC process, not the perl script that runs and then exits
  2. -localhost no so connections from other computers are accepted. This is only needed if you are not using ssh tunneling.
  3. User=steve so tigervnc doesn't complain about HOME not defined in the environment and uses my settings for auth and X session. Note that this is a strictly single-user setup.

2022-07-22

Using SubShapeBinders and Assembly4

Assembly4 and SubShapeBinders make it possible to assemble Parts, then using their position in the assembly to generate new Parts. However because new Parts depend on Parts in the assembly, if you try to add them to the assembly FreeCAD 0.19 will complain about cyclic redundancy, which makes sense.

To get around this, one method is to change the Bind Mode of the SubShapeBinders to "Disabled" which copies the geometry but unbinds it from the base object.

2022-06-18

Kopia: Handling Directories That No Longer Exist

 Scenario

  1. There exists a policy that snapshots directory ABC 
  2. Directory ABC now no longer exists
  3. You want to keep existing snapshots
  4. You want to stop kopia from attempting to take any more snapshots

In order to achieve (4):

  1. Set the policy to manual only
  2. Disable inheritance from parent/global policy

(2) is required as otherwise I find ABC's policy somehow inheriting the scheduling settings from the parent/global policy and thus is scheduled even though manual snapshots only is set.

To carry out the above steps:

  • KopiaUI: 
    • Edit Policy 
      • Scheduling > Manual Snapshots Only 
      • Other > Disable Parent Policy Evaluation
  • CLI: set policy --manual --inherit false

2022-03-05

Adventures with Astroberry/KStars/Ekos

In astrophotography the combination of a goto mount, a camera and plate-solving is a powerful one. It allows you to do all kind of neat things, like polar-alignment without having a clear view of the south, extremely accurate goto functionality, and automated capture of multiple predefined targets.

There are roughly speaking three popular single-board-commputers (SBCs) that facilitate this: ASIAIR PRO, Stellarmate and and astroberry, in increasing order of technical complexity and reduction in price, with astroberry being free. Both Stellarmate and astroberry use the same underlying software, namely kstars/ekos and INDI Library.

Since both ASIAR and Stellarmate devices were out of stock, but Raspberry Pi 4s (rpi) aren't, I opted to try astroberry. Installation was straightforward, everything worked as advertised. To ease mounting I designed and printed an accessory collar that allows the rpi to be attached and removed easily.

Raspberry Pi 4 in custom case mounted the accessory collar.


The first issue with astroberry, and possibly also with Stellarmate, is that control is done via a remote desktop session over VNC. This isn't a problem other than the fact we are forced to use a desktop software on mobile device with very different aspect ratios and interaction paradigms. The end result is a very janky user experience. There are also concerns that, as the rpi uses passive cooling, it will quickly be thermally throttled, further degrading user experience unless one adds a cooling fan and heatsink.

So the next evolution is to run only the INDI server on the rpi and run kstars and ekos on a laptop, which provides a native desktop experience appropriate to the device and moves any heavy processing onto a more capable platform. This works, but does take away the main advantages of having a SBC, namely a small and light setup.

The next issue that arises is that of transfer time. Running kstar/ekos on a laptop means images captured by the rpi must be transferred before any additional algorithms, such as plate-solving, can run. With my Canon 800D it takes up to 15 seconds to transfer a single raw file over the built-in wireless. This is an unacceptable long amount of time.

The logical step is then to do away with the rpi altogether and run everything on the laptop I am using anyway. In this case a lenovo X220 with extended batteries. This works well, with transfer times of approximately one second.

As an aside, the default ekos layout doesn't do well on a display that is only 768px high, with many UI elements in the capture tab unusable. Setting ekos to have tabs on the left as opposed to at the top resolved this issue.

The current setup is thus:

  1. Canon 800D with 200mm F/2.8L II
  2. SmartEQ Pro+
  3. Lenovo X220 with KStars/EKOS/INDI

Now if only the clouds would go away I can actually see how this performs in anger.

2022-02-23

Astrophotography Notes: ISO, Light Pollution, Histograms

ISO

I, like many people, thought that lower ISO means less noise. However this isn't true, at least not for modern camera. As a first order approximation, my understanding is that the signal path looks something like this:

  1. Photons impact the sensor
  2. Sensor generates electrical signal
  3. ISO amplifier amplifies the signal
  4. Readout electronics converts the signal into digital numbers
  5. digital numbers recorded as a RAW file

Generally speaking there are two major source of noise:

  1. Random noise inherent in the photons arriving at the sensor
  2. Noise generated by readout electronics and quantisation noise in the ADC
    1. aka readout-noise 

We can express the signal-to-noise ration (SNR) of the RAW image as follows:

RAW_SNR = (ISO_gain * S) / (ISO_gain * N + N_readout)

Where S and N is the light signal and noise, and N_readout is the readout-noise. ISO_gain is the gain  corresponding to the ISO value, but not the ISO value itself. e.g. ISO 100 doesn't not mean gain of 100.

Generally speaking S >> N, that is the light signal itself has good SNR. 

In low light situations where S of a similar magnitude to N_readout, if we use a low ISO such that ISO_gain is 1, then 

RAW_SNR ~= S / N_readout

as N_readout >> N

By increasing the ISO such (ISO_gain * N) >> N_readout,

RAW_SNR = (ISO_gain * S) / (ISO_gain * N + N_readout) 

 ~= (ISO_gain * S) / (ISO_gain * N) 

~= S / N

That is, we approximately recover the original SNR of the signal which is the best that can be asked for without additional processing.

This bears out in practice. I took a series of images in low light with the same aperture and shutter speed, changing only the ISO. Each image was then digitally processed in darktable with exposure compensation so images are of the same brightness (i.e. 1 extra step of exposure for each reduction in ISO):



The improvement in image quality at ISO 6400 compared to ISO 400 is significant. 

Light Pollution

With such high ISO, when I take pictures in my backyard in Bortle 7 skies, I get images that appear overexposed due to the high background brightness:

Single exposure, high background brightness

Histogram, plenty of room.


Previously I have viewed such images as "unusable". However upon looking at the histogram, it is clear that despite the background brightness, the majority of the signal is properly captured without clipping. Indeed, when post-processed in sirl, the final image does not suffer from the background glow:

So be not afraid of using high ISO in light polluted skies - as long as the histogram is not clipping the signal is there and can be recovered.


2022-02-19

GDK/GTK Signal Handling Notes

 I recently tried my hands at implementing motion event support for canvas widgets in Toga on Linux, which uses GTK3 as its GUI back end. In the process I had learn to navigate GDK's event handling mechanics and documentation. This post is a summary of notes I made during the process.

Basics

GTK widgets general signals, which you can be notified of if you connect() to the widget by specifying
  1. the name of the signal
  2. callback to invoke when the event occurs

Event Names

To find out what signals a widget emits you naturally look at the widget's documentation, e.g. for DrawingArea. However you will quickly realise not all signals are listed. The list does not include signals such as "draw" which is mentioned in python-gtk-3's documentation. The actual list of inherited signals can only be seen in the documentation for Widget. Why the disparity? No idea.

Event Mask

Not that widgets do not emit all possible signals, there is an EventMask bitfield that determines which signals a widget will emit. It is not enough to simply connect() to a signal - you must also enable those events you wish to receive if not enabled by default via gtk_widget_add_event(). Note that the relationship between signal names defined and the event-mask required is not explicitly defined. It comes down to guessing the how they are related and trial and error.



2022-02-09

Verifying GoTo Mount Settings

I recently got a iOptron SmartEQ Pro+ mount and while setting it up I was confused over the time. Should it be local time and then I set the UTC offset in minutes? What do I do with the DST setting?

One way I found of checking whether my settings is correct is

  1. Look up the altitude and azimuth coordinates of a bright star like Sirius using a planetarium app
  2. Command the mount to slew to that star
  3. On arrival the altitude and azimuth coordinates are displayed on the handset
  4. Compare the coordinates shown on the handset vs coordinates obtained in step (1). They should match to at least the degree. If not the time and location setting is wrong.
  5. Tweak settings and repeat from (2) onwards until there is at least degree level agreement between the mount and reference altitude and azimuth coordinates.

2022-01-02

Goodbye SpiderOak

I have been using SpiderOak One since Edward Snowden recommended it in 2013, almost a decade ago. I have over 1.5T of deduplicated data spread across 6 or so devices. This year, in 2022, I will not be renewing my subscriptions.

The main issues:

  1. Lack of updates: the last update to SpiderOak One was in 2019-09. All recent updates have been for CrossClave. It is evident that One is no longer a priority.
  2. Lack of speed: since day one (ha!) One has always been slower than competitors like Dropbox. In the beginning it wasn't so bad and one must makes some tradeoffs for the security offered. However in recent years this has not improved and setting up a new device takes days if it ever completes the initial synchronisation. Once setup the application is bloated, slow, generates massive amounts of local metadata.
  3. Lack of control: One installations will randomly stall on sync or update because _another_ device has  gone bad. This is not a great situations when I have devices spread across the city.

To replace SpiderOak One I am currently using:

  1. Kopia for backup with de-duplication and client-side encryption.  The target is backblaze b2.
  2. Sync.com for file sync. Note that Sync.com doesn't provide the same security guarantees as Kopia or SpiderOak, but I am willing to trade that for a sync solution that is fast, quick and affordable.
    1. There is no native linux client but the Sync.com client runs decently under wine and the performance on my slowest machines are acceptable.