Practical experiences from every day IT

Category: Linux

  • Grafana replaces datasources from older imported dashboards with the default datasource

    … potentially rewriting the dashboard json code in sometimes unexpected and unwanted ways.

    When importing some old dashboard that used “null” as datasource, newer instances of Grafana do replace this with the default datasource. In my case, where datasources such as Graphite and Prometheus are mixed, this can be very annoying when importing a dashboard that was made for Prometheus.

    One solution can be to bulk-change the occurrences of

    "datasource": null

    with sed:

    sed -i 's/"datasource": null/"datasource": {"type": "prometheus", "uid": "${DS_PROMETHEUS}"}/g' dashboard.json

    This declares the datasource to be a variable datasource to be chosen by a custom annotation.

    It also prevents Grafana from removing correct Prometheus queries for panels, which it does, if it things that the datasource is a Graphite datasource.

  • Kingston SUV500 with Dell PERC H730

    Both the PERC H730 and the Kingston SUV500 are really old hardware. Not only that, they also served two completely different markets: Enterprise versus Consumer. But that should not be a reason for the problem I describe next: There is some strange incompatibility running the Kingston SUV500 with a DELL PERC RAID-Controller (both in RAID- and HBA-mode).

    I recently had acquired some old server hardware to solve the recent, sometimes unpredictable, pricing changes for virtual and dedicated servers. Since the R330’s have plenty of free slots, I have had someone print me a couple of DELL 2,5 inch caddys and off we went to install the machines.

    After the initial installation on the SUV500 as a OS-disk, I started to encounter strange anomalies:

    • getting stuck at “Booting from Harddisk C:”
    • getting stuck at PERC controller initialization “FW initializing 33%”
    • slow performance after controller cache was used

    Forth and back I tried to play with the cache settings as well as HBA-mode, with no success. Updating the SUV500’s firmware on a Windows box that I luckily had around, with no success. Then I researched the problem a little and found that I am not the only one with problems in that constellation.

    And despite the forum members in some forums spreading useless, even stupid misinformation about the interoperability of enterprise- and consumer-hardware, the core truth stays the same: Somewhere in the firmware of at least one of the components there is something wrong. The complexity of reproducing and analyzing the problem deeper might make it impossible or at least very unfeasible for me.

    I went and installed another 2,5 inch SSD for the OS, some Samsung PM8xx model with 256GB. All went fine.

  • ffmpeg for recording from v4l2

    Every now and then some old VHS pops up after cleaning old storages, shelfes, workshops. And I am by far not a professional VHS digitizing service, but some VHS’s are really worth preserving to me, so I have a cheap VHS player and a Hauppauge USB-Live2 analogue USB capture device.

    A very simple way to capture the video and store it, would be:

    ffmpeg -f v4l2 -framerate 25 -i /dev/video0 -t 03:10:00 -f alsa -i plughw:2,0 -c:v libx264 -vf format=yuv420p -preset ultrafast -crf 23 -c:a aac -b:a 192k output2.mp4

    One could ofc adjust some parameters. In my case this was running on old hardware, where everything more CPU-bound than libx264 with the “ultrafast” preset would congest the CPU too much, leading to frame drops.

    The point here was also never to store in archive quality, for that I would need another VHS playback device with a TBC (Time Base Corrector) to decrease the horizontal jitter in the VHS, anyway. They are expensive. Or I would have to enter the whole new world of vhs-decode. Maybe one day.

  • Simulating 1024 bit dh params for sftp

    Sometimes I work with very old IT server infrastructure, but this should not be the topic of today. We focus on the client side today, where it happens from time to time that companies and people from various fields, such as old client software based on Java 5, 6, 7, or 8, are bound to this kind of legacy for other reasons.

    As an operator of services such as SFTP, this really is not ideal. You either have to exclude such clients to maintain an up-to-date server infrastructure, or include them and provide some form of backwards compatibility.

    To test that compatibility, you would need to either have all clients available that your peers use or at least be able to simulate them. I happened to have issues with clients that require, by nowadays’ standards, very short Diffie-Hellman parameters to be supplied by the server. Java 5, 6, 7, 8 are limited to 1024 bit of length.

    (more…)

  • rsync: File name too long (36) when moving data towards Synology NAS

    Likely this is because your filenames are too long, over 255 bytes. However, if you have enabled encryption on your shares in Synology, that limit is lowered to 143 bytes, which is much easier to hit.

    Longer explanation

    eCryptfs, which Synology uses for encrypted shares, must keep encrypted filenames within the underlying filesystem’s 255-byte limit, but encryption adds padding and metadata and then expands further due to Radix-64 Uuencoding. Because these transformations increase size significantly, the original plaintext-filename is limited to about 143 bytes to ensure the final encoded name never exceeds 255 bytes. The recommendation of the developers of eCryptfs actually suggests a limit of 140 bytes.

    I specifically noticed this problem when moving around data from an encrypted TrueNAS setup to an encrypted Synology share via rsync+ssh. Because TrueNAS uses ZFS’ encryption feature, it does not have this constraint.

    Special attention required when copying with rsync -v

    When copying with rsync -v, the output is very verbose and it is easy to miss copy errors due to long filenames. Taking this further, there is a high chance for data loss when no special attention is applied. If you need to rsync -v, make sure to also do a second run without -v to really only see the errors listed.

    Solution 1

    Create a new share without encryption and copy over all files with the tool of your choice. Then merge your data towards the Synology NAS’ unencrypted share.

    Solution 2

    Shorten your filenames.

  • Alpine 3.23 mkimage.sh: ERROR: –usermode not allowed as root

    Recently I decided to upgrade a custom Alpine ISO generator that I run to the latest version of Alpine Linux, 3.23. The task is fairly easy, but required some special attention, because the first ISO build attempt failed:

    sh aports/scripts/mkimage.sh --tag "test-3.23-0.1" --outdir /export --arch x86_64 --repository https://dl-cdn.alpinelinux.org/alpine/v3.23/main --repository https://dl-cdn.alpinelinux.org/alpine/v3.23/community --profile myprofile
ERROR: --usermode not allowed as root

    After some research I found that this is related to the updated version of apk, the package manager used in Alpine Linux. Version 3.x of apk brings a fair amount of new features and some slightly breaking changes such as this one.

    (more…)