2022-06-18

Kopia: Handling Directories That No Longer Exist

 Scenario

  1. There exists a policy that snapshots directory ABC 
  2. Directory ABC now no longer exists
  3. You want to keep existing snapshots
  4. You want to stop kopia from attempting to take any more snapshots

In order to achieve (4):

  1. Set the policy to manual only
  2. Disable inheritance from parent/global policy

(2) is required as otherwise I find ABC's policy somehow inheriting the scheduling settings from the parent/global policy and thus is scheduled even though manual snapshots only is set.

To carry out the above steps:

  • KopiaUI: 
    • Edit Policy 
      • Scheduling > Manual Snapshots Only 
      • Other > Disable Parent Policy Evaluation
  • CLI: set policy --manual --inherit false

2022-03-05

Adventures with Astroberry/KStars/Ekos

In astrophotography the combination of a goto mount, a camera and plate-solving is a powerful one. It allows you to do all kind of neat things, like polar-alignment without having a clear view of the south, extremely accurate goto functionality, and automated capture of multiple predefined targets.

There are roughly speaking three popular single-board-commputers (SBCs) that facilitate this: ASIAIR PRO, Stellarmate and and astroberry, in increasing order of technical complexity and reduction in price, with astroberry being free. Both Stellarmate and astroberry use the same underlying software, namely kstars/ekos and INDI Library.

Since both ASIAR and Stellarmate devices were out of stock, but Raspberry Pi 4s (rpi) aren't, I opted to try astroberry. Installation was straightforward, everything worked as advertised. To ease mounting I designed and printed an accessory collar that allows the rpi to be attached and removed easily.

Raspberry Pi 4 in custom case mounted the accessory collar.


The first issue with astroberry, and possibly also with Stellarmate, is that control is done via a remote desktop session over VNC. This isn't a problem other than the fact we are forced to use a desktop software on mobile device with very different aspect ratios and interaction paradigms. The end result is a very janky user experience. There are also concerns that, as the rpi uses passive cooling, it will quickly be thermally throttled, further degrading user experience unless one adds a cooling fan and heatsink.

So the next evolution is to run only the INDI server on the rpi and run kstars and ekos on a laptop, which provides a native desktop experience appropriate to the device and moves any heavy processing onto a more capable platform. This works, but does take away the main advantages of having a SBC, namely a small and light setup.

The next issue that arises is that of transfer time. Running kstar/ekos on a laptop means images captured by the rpi must be transferred before any additional algorithms, such as plate-solving, can run. With my Canon 800D it takes up to 15 seconds to transfer a single raw file over the built-in wireless. This is an unacceptable long amount of time.

The logical step is then to do away with the rpi altogether and run everything on the laptop I am using anyway. In this case a lenovo X220 with extended batteries. This works well, with transfer times of approximately one second.

As an aside, the default ekos layout doesn't do well on a display that is only 768px high, with many UI elements in the capture tab unusable. Setting ekos to have tabs on the left as opposed to at the top resolved this issue.

The current setup is thus:

  1. Canon 800D with 200mm F/2.8L II
  2. SmartEQ Pro+
  3. Lenovo X220 with KStars/EKOS/INDI

Now if only the clouds would go away I can actually see how this performs in anger.

2022-02-23

Astrophotography Notes: ISO, Light Pollution, Histograms

ISO

I, like many people, thought that lower ISO means less noise. However this isn't true, at least not for modern camera. As a first order approximation, my understanding is that the signal path looks something like this:

  1. Photons impact the sensor
  2. Sensor generates electrical signal
  3. ISO amplifier amplifies the signal
  4. Readout electronics converts the signal into digital numbers
  5. digital numbers recorded as a RAW file

Generally speaking there are two major source of noise:

  1. Random noise inherent in the photons arriving at the sensor
  2. Noise generated by readout electronics and quantisation noise in the ADC
    1. aka readout-noise 

We can express the signal-to-noise ration (SNR) of the RAW image as follows:

RAW_SNR = (ISO_gain * S) / (ISO_gain * N + N_readout)

Where S and N is the light signal and noise, and N_readout is the readout-noise. ISO_gain is the gain  corresponding to the ISO value, but not the ISO value itself. e.g. ISO 100 doesn't not mean gain of 100.

Generally speaking S >> N, that is the light signal itself has good SNR. 

In low light situations where S of a similar magnitude to N_readout, if we use a low ISO such that ISO_gain is 1, then 

RAW_SNR ~= S / N_readout

as N_readout >> N

By increasing the ISO such (ISO_gain * N) >> N_readout,

RAW_SNR = (ISO_gain * S) / (ISO_gain * N + N_readout) 

 ~= (ISO_gain * S) / (ISO_gain * N) 

~= S / N

That is, we approximately recover the original SNR of the signal which is the best that can be asked for without additional processing.

This bears out in practice. I took a series of images in low light with the same aperture and shutter speed, changing only the ISO. Each image was then digitally processed in darktable with exposure compensation so images are of the same brightness (i.e. 1 extra step of exposure for each reduction in ISO):



The improvement in image quality at ISO 6400 compared to ISO 400 is significant. 

Light Pollution

With such high ISO, when I take pictures in my backyard in Bortle 7 skies, I get images that appear overexposed due to the high background brightness:

Single exposure, high background brightness

Histogram, plenty of room.


Previously I have viewed such images as "unusable". However upon looking at the histogram, it is clear that despite the background brightness, the majority of the signal is properly captured without clipping. Indeed, when post-processed in sirl, the final image does not suffer from the background glow:

So be not afraid of using high ISO in light polluted skies - as long as the histogram is not clipping the signal is there and can be recovered.


2022-02-19

GDK/GTK Signal Handling Notes

 I recently tried my hands at implementing motion event support for canvas widgets in Toga on Linux, which uses GTK3 as its GUI back end. In the process I had learn to navigate GDK's event handling mechanics and documentation. This post is a summary of notes I made during the process.

Basics

GTK widgets general signals, which you can be notified of if you connect() to the widget by specifying
  1. the name of the signal
  2. callback to invoke when the event occurs

Event Names

To find out what signals a widget emits you naturally look at the widget's documentation, e.g. for DrawingArea. However you will quickly realise not all signals are listed. The list does not include signals such as "draw" which is mentioned in python-gtk-3's documentation. The actual list of inherited signals can only be seen in the documentation for Widget. Why the disparity? No idea.

Event Mask

Not that widgets do not emit all possible signals, there is an EventMask bitfield that determines which signals a widget will emit. It is not enough to simply connect() to a signal - you must also enable those events you wish to receive if not enabled by default via gtk_widget_add_event(). Note that the relationship between signal names defined and the event-mask required is not explicitly defined. It comes down to guessing the how they are related and trial and error.



2022-02-09

Verifying GoTo Mount Settings

I recently got a iOptron SmartEQ Pro+ mount and while setting it up I was confused over the time. Should it be local time and then I set the UTC offset in minutes? What do I do with the DST setting?

One way I found of checking whether my settings is correct is

  1. Look up the altitude and azimuth coordinates of a bright star like Sirius using a planetarium app
  2. Command the mount to slew to that star
  3. On arrival the altitude and azimuth coordinates are displayed on the handset
  4. Compare the coordinates shown on the handset vs coordinates obtained in step (1). They should match to at least the degree. If not the time and location setting is wrong.
  5. Tweak settings and repeat from (2) onwards until there is at least degree level agreement between the mount and reference altitude and azimuth coordinates.

2022-01-02

Goodbye SpiderOak

I have been using SpiderOak One since Edward Snowden recommended it in 2013, almost a decade ago. I have over 1.5T of deduplicated data spread across 6 or so devices. This year, in 2022, I will not be renewing my subscriptions.

The main issues:

  1. Lack of updates: the last update to SpiderOak One was in 2019-09. All recent updates have been for CrossClave. It is evident that One is no longer a priority.
  2. Lack of speed: since day one (ha!) One has always been slower than competitors like Dropbox. In the beginning it wasn't so bad and one must makes some tradeoffs for the security offered. However in recent years this has not improved and setting up a new device takes days if it ever completes the initial synchronisation. Once setup the application is bloated, slow, generates massive amounts of local metadata.
  3. Lack of control: One installations will randomly stall on sync or update because _another_ device has  gone bad. This is not a great situations when I have devices spread across the city.

To replace SpiderOak One I am currently using:

  1. Kopia for backup with de-duplication and client-side encryption.  The target is backblaze b2.
  2. Sync.com for file sync. Note that Sync.com doesn't provide the same security guarantees as Kopia or SpiderOak, but I am willing to trade that for a sync solution that is fast, quick and affordable.
    1. There is no native linux client but the Sync.com client runs decently under wine and the performance on my slowest machines are acceptable.

2021-11-24

Quick Survey of Some Lightweight Linux Distributions

On an ASUS E203M

Busenlabs Lithium

Installed fine but kernel (4.9?) would immediately shutdown after decrypting the root volume due to incorrect thermal readings. Was not able to disable this via thermal.nocrt=1 boot param

 

AntiX 21

Live system did not have working mouse. Did not proceed to installation as I didn't want to be stuck with having to fix the mouse issue.

 

bodhi linux HWE 6.0.0

Mouse works in the live system, I like the Enlightenment lineage. No GUI setting to disable tap-to-click on the touchpad, resolved via synclient MaxTapTime=0 so not a deal breaker. However despite being lightweight there was noticable GUI lag when running the installer. 

 

Q4OS Gemini, Trinity

I like the idea of Trinity Desktop and they are using a decently recent kernel as well (5.10). Installed fine but wifi doesn't connect to the 5G network. Worked fine on 2.4G. Had an annoying issue where Alt-Space didn't work, giving me ISO_Next_Group. Fixed it by setting via xkb options to use "both ctrl at the same time" to switch languages which then allowed alt-space to work as intended.

Systems feels really snappy and the retro look is not bad. Sticking with this one for now.

2019-12-14

How *Not* to Play with Programmable LED Strips

You Need That Resistor

That resistor on the data line? Yeah you need it. Otherwise you might blow an LED which renders the entire rest of the strip useless. If that happens cut out the first LED and solder the power and data lines directly to the second LED.

You Want Shorter, Thicker, Wires

Using jumper wires, of the 300-for-$5 variety is fine for a few LEDs. If however you have 300 LED drawing around 1A then those thin wires have non-negligible voltage drop. For me this was on the order of 0.7V or more. This means even though my power supply was putting out 5V the LED strip was only seeing 4.3V, causing flickering of the LEDs.

For any piece of wire the voltage drops increases with current simply because of V=IR. Thicker wires have less resistance per unit length and therefore less voltage drop per unit length.

2019-11-12

The Subtle Poison of Religion

 On the 12th of November 2019, Nationals MP Barnaby Joyce said on TV that:
 I acknowledge that the two people who died were most likely people who voted for the Green party, so I am not going to start attacking them. That's the last thing I want to do.
Why does the politics of the deceased matter and why would Mr. Joyce make such a speculation? I can't help but think that Mr. Joyce did it because, on some level, he thinks those people deserve it. I think this is the subtle poison of religion: that those who do not believe as I do deserve less than I do.

This idea is widespread across all variants of Christianity: you are saved if you believe in God and Jesus and the Holy Spirit but damned if you don't. You deserve salvation only if you Believe. It is this idea that motivates the Evangelicals: they must Save you by spreading their Belief so you, too, deserve salvation. It is this idea that underpins the prosperity gospel: those who Believe is deserving of wealth and therefore the harder you Believe, whether by praying harder or donating more to the Church, the more you deserve wealth. Conveniently if you are wealthy then obviously it is because you deserve it due to your faith.

Once learned and integrated into one's world view via religious instruction, this idea, that those who do not believe as I do deserve less, poisons the lens through which an individual interacts with the world. When a core part of your identify requires you to accept that, on the basis of belief alone, not everyone is equal, then it is a small step to thinking that those who hold different political beliefs deserve less then those whose beliefs align with yours. Maybe they deserve less sympathy. Maybe they deserve less government support. Maybe they deserve less compassion or concession. Whatever the case may be, on the basis of their political belief, they are Other and they are Undeserving -- and in politics this is a problem.

Some would claim that it is possible to separate one's religious beliefs from one's political actions. I reject such claims in the same way I reject any claims one can be racist and still be a fair judge. Some would claim that some people think thusly even without religion and I would agree. However some religions requires such thinking. Some would say not all religions features undeserving unbelievers and claim my title is inaccurate and I would concede their point but also point out that the major religions do not fall within this category.

2019-06-24

3D Printing at Home

FlashForge Creator Pro

I finally got my own 3D printer, a FlashForge Creator Pro, during the EOFY sale. My choices were between the Creator Pro or the Up Mini 2 ES with the build volume of the Creator Pro being the deciding factor.

Had bit of a rocky start with bed leveling but nothing a bit of patience won't fix. It prints reliably now though I have plans for a semi-assisted bed leveling jig.

Designs

Over the past week I have worked on and off on an iPad mini holder to position it above my monitor so I can use it as a status screen or as a monitor to watch twitch while I am doing menial tasks.



I also made a small headphone hanger to save a bit of space.



Both of these designs were made in Fusion360, my first with the software. Normally I use blender because that's what I comfortable with but the Fusion brings a lot to the table and I will be using it going forward. It doesn't hurt that experience and skills with Fusion will be useful in an industrial setting.

Next on the list is some replacement end-stops for a wall hanging and a keypad bruteforcer.

2018-08-21

T102HAAS.303 Considered Harmful For Linux

T102HAAS.303 Breaks Suspend


After upgrading my Asus Transformer Mini's BIOS to T102HAAS.303 suspend was completely broken. Closing the "lid" would cause some kind of suspend that cannot be disabled in software and from which Linux 4.15.0 cannot successfully resume from (actually it does resume but the display will stay blank).

This behaviour cannot be disabled in software by editing logind.conf or acpi-support or any of the solutions found on the web about disabling lid action. Messing about with acpi_osi didn't help either.

Downgrading from T102HAAS.303


If you are reading this after upgrading to T102HAAS.303 you probably want to downgrade. Unfortunately the firmware that ships with the computers, version 202, cannot be found on the Asus website. You can however find the 300 firmware if you click on "View all downloads" under the BIOS section. Download this file and put it on a FAT32 formatted USB drive then:

  1. Reboot, hold then ESC to enter Setup
  2. Find the Easy Update entry and press ENTER, starting the easy update program
  3. Navigate to your USB drive, it will be FS0 or FS1
  4. Locate the T102HAAS.303 file and press ENTER
  5. When the dialogue complains that the build date is too old type the word risky on the keyboard
  6. Select Yes on the dialogue that appears after
  7. Wait for the computer to reboot

The key to this process is the word risky which is an undocumented feature that by-passes the build date check, allowing you to downgrade. You just type this in when the error dialogue appears, there will be no text box or anything.

Re-enabling Hibernation


Downgrading will re-enable Secure Boot, which disables hibernation on Linux. Just disable it and hibernation will work again.

2018-08-06

Ubuntu 18.04 on Asus Transformer Mini (T102H)

Installation


Create a USB drive with a single FAT32 partition then extract the installation ISO into it using 7z:

7z x /path/to/ISO -o/path/to/usb

Note that there is no space between -o and the following path.

Plug in the USB drive then turn on the T102H while pressing the ESC key. This will allow you to enter the settings menu and allow you to boot from USB. Select installation and follow the instructions until asked to reboot.

Xorg Setup


An immediate problem is that the display is portrait by default. Additionally the modesetting driver is used which has worse performance than the intel driver. Fix both of these issue by placing the following file in /usr/share/X11/xorg.conf.d/99-transformer.conf:

Section "Device"
 Identifier "i915"
 Driver "intel"
 Option "TearFree" "true"
 Option "DRI" "2"
 Option "Backlight" "intel_backlight"
EndSection

Section "Screen"
 Identifier "Screen0"
 Monitor "BuiltInLCD"
EndSection

Section "Monitor"
 Identifier "BuiltInLCD"
 Option "Rotate" "right"
EndSection

Section "InputClass"
 Identifier "BuiltInTouchscreen"
 MatchProduct "ELAN22A6:00 04F3:22A6"
 Option "TransformationMatrix" "0 1 0 -1 0 1 0 0 1"
EndSection

This sets the display driver to intel, rotates the monitor, and also transforms touchscreen inputs to match the landscape orientation.

i915

To further improve performance place the following into /etc/modprobe.d/i915.conf
options i915 enable_fbc=1 enable_guc=3 enable_psr=1 disable_power_well=0 semaphores=1

Backlight


Out of the box backlight doesn't work because of a module loading order issue [96571]. The easiest fix, found in the bug report thread, is to use dracut to remove the i915 module from initramfs so it loads after the PWM modules.

sudo apt install dracut
sudo dracut -f --omit-drivers="i915"
sudo update-grub

This issue may be fixed in a later release as the issue has been reported to Ubuntu.

After rebooting /sys/class/backlight/intel_backlight/ should exist, indicating that backlight control is available.

Keyboard Backlight Control


Unfortunately while we can now control the backlight using programs like xbacklight, the screen brightness keys (fn+F5 and fn+F6) don't work. We can get around this by binding control-F5 and control-F6 to trigger xbacklight -dec 20 and xbacklight -inc 20 respectively to decrement/increment screen brightness by 20% per keyboard press.

Airplane Mode


Similar to the backlight control buttons the airplane mode button also doesn't work. The following script uses rfkill to toggle bluetooth and wifi on/off, effectively implementing airplane mode in software. I haven't found a way to disable bluetooth and wifi in hardware.

#!/bin/bash

BLUETOOTH_OFF_BY_DEFAULT=1

n_devices="$(rfkill -n | wc -l)"
n_blocked="$(rfkill -n | grep -w blocked | wc -l)"

echo "$n_devices devices, $n_blocked blocked"

if [ "$n_blocked" == "$n_devices" ]; then
  rfkill unblock bluetooth wlan

  if [ "$BLUETOOTH_OFF_BY_DEFAULT" == "1" ]; then
    rfkill block bluetooth
  fi
else
  rfkill block bluetooth wlan
fi

Power Saving


It is a good idea to install tlp to improve the running time

TODOs:

  • Script and keybindings to rotate the screen and touchscreen
  • Pressure level with the pen