Planet Debian

Subscribe to Planet Debian feed
Planet Debian - https://planet.debian.org/
Updated: 2 hours 36 min ago

Utkarsh Gupta: GSoC Phase 2

16 July, 2020 - 00:00

Hello,

In early May, I got selected as a Google Summer of Code student for Debian to work on a project which is to write a linter (an extension to RuboCop).
This tool is mostly to help the Debian Ruby team. And that is the best part, I love working in/for/with the Ruby team!
(I’ve been an active part of the team for 19 months now :))

More details about the project can be found here, on the wiki.
And also, I have got the best mentors I could’ve possibly asked for: Antonio Terceiro and David Rodríguez 💖

So, the program began on 1st June and I’ve been working since then. I log my daily updates at gsocwithutkarsh2102.tk.
The blog for the first part of phase 1 can be found here and that of second part of phase 1 can be found here.

Whilst the daily updates are available at the above site^, I’ll breakdown the important parts here:

  • After the, what I’d like to call, successful Phase 1, whilst using this extension on GitLab’s omnibus-gitlab repository, I discovered a bug (as reported via issue #5), which basically threw the following error:

    An error occurred while Packaging/GemspecGit cop was inspecting /home/utkarsh/github/omnibus-gitlab/files/gitlab-cookbooks/gitlab/recipes/bootstrap_disable.rb.
    To see the complete backtrace run rubocop -d.

    …which was not good.

  • This bug was a false-negative and the fix turned out to be simple, which was raised via PR #6.
    Since the extension was now working fine against omnibus-gitlab (which is…quite huge!), I rolled out a v0.1.1 release! 🎉

  • Next, I implemented the 2nd cop, fixing require and require_relative calls which map from spec(s)/test(s) to the lib directory.
    This was raised via a WIP PR #4.

  • We had our meeting where we discussed that it would rather make sense to split these cops into two parts and also, thanks to David’s elaborative comment on the situation.
    With this, we decided to split the cops into two.

  • Then I worked on:

  • At this point, we hit yet another obstacle. Correctly determining the root directory of a project at runtime. This is…tricky. Not impossible (of course) but tricky.
    So my “homework” was to find such a thing that does that by default.

  • For the next 4 days, I tried to find something that could do this bit. But unfortunately, I couldn’t.
    So I wrote one myself. I wrote get_root, which solves this problem.
    The only thing that one needs to take care while using get_root is that it has to be a git repository. That’s a…trade-off.

  • In the next meeting, we discussed that this is a bit of an overkill. get_root should’ve been written as a helper function and not as another library, which David and Antonio pointed out correctly. But it was already too late :P
    I had already made a v 0.1.0 release.
    (So in case you’re a Rubyist and want to find the root directory of a git repository, consider using this :P)
    Besides, Antonio also pointed out that it should take another arguement as to from point from where the file is being inspected. Hm, unsure how to do that..

  • Meanwhile, I exported get_root as a helper function, David solved all the problem altogether at once via rubocop’s PR #8314.
    This introduced a new (public) method #project_root which superseeded get_root and one could now get the root directory via RuboCop::ConfigLoader.project_root. Ain’t he amazing!? \o/

  • This also means, I reverted those changes altogether and tweaked my WIP PR to inculcate these changes via commit @b06a8f86.
    However, the specs fail. But that doesn’t mean that the changes aren’t correct :P
    They are pretty much right and working fine. To make that sure, I locally installed this library and used on other projects to make sure that it indeed is working alright, as it should! \o/

  • And here I am on the 15th day :)

Well, the best part yet?
rubocop-packaging is being used by batalert, arbre, rspec-stubbed_env, rspec-pending_for, ISO8601, get_root (😛), gir_ffi, linter, and cucumber-rails.

Whilst it has been a lot of fun so far, my plate has started to almost…overflow. It seems that I’ve got a lot of things to work on (and already things that are due!).
From my major project, college *stuff to my GSoC project, Debian (E)LTS, and a lot *more.

Thanks to Antonio for helping me out with *other things (which maps back to his sayings in Paris \o/).

Until next time.
:wq for today.

Antoine Beaupré: Goatcounter analytics in ikiwiki

15 July, 2020 - 09:07

I have started using Goatcounter for analytics after reading this LWN article called "Lightweight alternatives to Google Analytics". Goatcounter has an interesting approach to privacy in that it:

tracks sessions using a hash of the browser's user agent and IP address to identify the client without storing any personal information. The salt used to generate these hashes is rotated every 4 hours with a sliding window.

There was no Debian package for the project, so I filed a request for package and instead made a fork of the project to add a Docker image.

This page documents how Goatcounter was setup from there...

Server configuration
  1. build the image from this fork

    docker build -t zgoat/goatcounter .
    
  2. create volume for db:

    docker volume create goatcounter
    
  3. start the server:

    exec docker run --restart=unless-stopped --volume="goatcounter:/home/user/db/" --publish 127.0.0.1:8081:8080 --detach zgoat/goatcounter serve -listen :8080 -port 8080 -tls none
    
  4. apache configuration:

    <VirtualHost *:80>
                ServerName analytics.anarc.at
                Redirect / https://analytics.anarc.at/
                DocumentRoot /var/www/html/
        </VirtualHost>
    
    
    <VirtualHost *:443>
            ServerName analytics.anarc.at
            Use common-letsencrypt-ssl analytics.anarc.at
            DocumentRoot /var/www/html/
            ProxyPass /.well-known/ !
            ProxyPass / http://localhost:8081/
            ProxyPassReverse / http://localhost:8081/
            ProxyPreserveHost on
    </VirtualHost>
    
  5. add analytics.anarc.at to DNS

  6. create a TLS cert with LE:

    certbot certonly --webroot  -d analytics.anarc.at --webroot-path /var/www/html/
    

    note that goatcounter has code to do this on its own, but we avoid it to follow our existing policies and simplify things

  7. create site:

    docker run -it --rm --volume="goatcounter:/home/user/db/" zgoat/goatcounter create -domain analytics.anarc.at -email anarcat+rapports@anarc.at
    
  8. add to ikiwiki template

  9. rebuild wiki:

    ikiwiki --setup ikiwiki.setup --rebuild --verbose
    
Remaining issues
  • the :8080 port leaks in some places, namely in the "Site config" documentation
  • move to Docker Compose or podman instead of just starting the thing by hand
  • this is all super janky and should be put in config management somehow
  • remove "anarc.at" test site (the site is the analytics site, not the tracked site), seems like this is not possible yet
  • do log parsing instead of Javascript or 1x1 images?
  • compare with goaccess logs, probably in september
  • goatcounter monitor doesn't with sqlite
Fixed issues

Ian Jackson: MessagePack vs CBOR (RFC7049)

15 July, 2020 - 00:55

tl;dr: Use MessagePack, rather than CBOR.

Introduction

I recently wanted to choose a binary encoding. This was for a project using Rust serde, so I looked at the list of formats there. I ended up reading about CBOR and MessagePack.

Both of these are binary formats for a JSON-like data model. Both of them are "schemaless", meaning you can decode them without knowing the structure. (This also provides some forwards compatibility.) They are, in fact, quite similar (although they are totally incompatible). This is no accident: CBOR is, effectively, a fork of MessagePack.

Both formats continue to exist and both are being used in new programs. I needed to make a choice but lacked enough information. I thought I would try to examine the reasons and nature of the split, and to make some kind of judgement about the situation. So I did a lot of reading [11]. Here are my conclusions.

History and politics

Between about 2010 and 2013 there was only MessagePack. Unfortunately, MessagePack had some problems. The biggest of these was that it lacked a separate string type. Strings were to be encoded simply as byte blocks. This caused serious problems for many MessagePack library implementors: for example, when decoding a MessagePack file the Python library wouldn't know whether to produce a Python bytes object, or a string. Straightforward data structures wouldn't round trip through MessagePack. [1] [2]

It seems that in late 2012 this came to the attention to someone with an IETF background. According to them, after unsatisfactory conversations with MessagePack upstream, they decided they would have to fork. They submitted an Internet Draft for a partially-incompatible protocol [3] [4]. Little seemed to happen in the IETF until soon before the Orlando in-person IETF meeting in February 2013.[5]

These conversations sparked some discussion in the MessagePack issue tracker. There were long threads including about process [1,2,4 ibid]. But there was also a useful technical discussion, about proposed backward compatible improves to the MessagePack spec.[5] The prominent IETF contributor provided some helpful input in these discussions in the MessagePack community - but also pushed quite hard for a "tagging" system, which suggestion was not accepted (see my technical analysis, below).

An improved MessagePack spec resulted, with string support, developed largely by the MessagePack community. It seems to have been available in useable form since mid-2013 and was officially published as canonical in August 2013.

Meanwhile a parallel process was pursued in the IETF, based on the IETF contributor's fork, with 11 Internet-Drafts from February[7] to September[8]. This seems to have continued even though the original technical reason for the fork - lack of string vs binary distinction - no longer applied. The IETF proponent expressed unhappiness about MessagePack's stewardship and process as much as they did about the technical details [4, ibid]. The IETF process culminated in the CBOR RFC[9].

The discussion on process questions between the IETF proponent and MessagePack upstream, in the MessagePack issue tracker [4, ibid] should make uncomfortable reading for IETF members. The IETF acceptance of CBOR despite clear and fundamental objections from MessagePack upstream[13] and indeed other respected IETF members[14], does not reflect well on the IETF. The much vaunted openness of the IETF process seems to have been rather one-sided. The IETF proponent here was an IETF Chair. Certainly the CBOR author was very well-spoken and constantly talks about politeness and cooperation and process; but what they actually did was very hostile. They accused the MessagePack community of an "us and them" attitude while simultaneously pursuing a forked specification!

The CBOR RFC does mention MessagePack in Appendix E.2. But not to acknowledge that CBOR was inspired by MessagePack. Rather, it does so to make a set of tendentious criticisms of MessagePack. Perhaps these criticisms were true when they were first written in an I-D but they were certainly false by the time the RFC was actually published, which occurred after the MessagePack improvement process was completely concluded, with a formal spec issued.

Since then both formats have existed in parallel. Occasionally people discuss which one is better, and sometimes it is alleged that "yes CBOR is the successor to MessagePack", which is not really fair.[9][10]

Technical differences

The two formats have a similar arrangement: initial byte which can encode small integers, or type and length, or type and specify a longer length encoding. But there are important differences. Overall, MessagePack is very significantly simpler.

Floating point

CBOR supports five floating point formats! Not only three sizes of IEEE754, but also decimal floating point, and bigfloats. This seems astonishing for a supposedly-simple format. (Some of these are supported via the semi-optional tag mechanism - see below.)

Indefinite strings and arrays

Like MessagePack, CBOR mostly precedes items with their length. But CBOR also supports "indefinite" strings, arrays, and so on, where the length is not specified at the beginning. The object (array, string, whatever) is terminated by a special "break" item. This seems to me to be a mistake. If you wanted the kind of application where MessagePack or CBOR would be useful, streaming sub-objects of unknown length is not that important. This possibility considerably complicates decoders.

CBOR tagging system

CBOR has a second layer of sort-of-type which can be attached to each data item. The set of possible tags is open-ended and extensible, but the CBOR spec itself gives tag values for: two kinds of date format; positive and negative bignums; decimal floats (see above); binary but expected to be encoded if converted to JSON (in base64url, base64, or base16); nestedly encoded CBOR; URIs; base64 data (two formats); regexps; MIME messages; and a special tag to make file(1) work.

In practice it is not clear how many of these are used, but a decoder must be prepared to at least discard them. The amount of additional spec complexity here is quite astonishing. IMO binary formats like this will (just like JSON) be used by a next layer which always has an idea of what the data means, including (where the data is a binary blob) what encoding it is in etc. So these tags are not useful.

These tags might look like a middle way between (i) extending the binary protocol with a whole new type such as an extension type (incompatible with old readers) and encoding your new kind data in a existing type (leaving all readers who don't know the schema to print it as just integers or bytes or string). But I think they are more trouble than they are worth.

The tags are uncomfortably similar to the ASN.1 tag system, which is widely regarded as one of ASN.1's unfortunate complexities.

MessagePack extension mechanism

MessagePack explicitly reserves some encoding space for users and for future extensions: there is an "extension type". The payload is an extension type byte plus some more data bytes; the data bytes are in a format to be defined by the extension type byte. Half of the possible extension byte values are reserved for future specification, and half are designated for application use. This is pleasingly straightforward.

(There is also one unused primary initial byte value, but that would be rejected by existing decoders and doesn't seem like a likely direction for future expansion.)

Minor other differences in integer encoding

The encodings of integers differ.

In MessagePack, signed and unsigned integers have different typecodes. In CBOR, signed and unsigned positive integers have the same typecodes; negative integers have a different set of typecodes. This mean that a CBOR reader which knows it is expecting a signed value will have to do a top-bit-set check on the actual data value! And a CBOR writer must check the value to choose a typecode.

MessagePack reserves fewer shortcodes for small negative integers, than for small positive integers.

Conclusions and lessons

MessagePack seems to have been prompted into fixing the missing string type problem, but only by the threat of a fork. However, this fork went ahead even after MessagePack clearly accepted the need for a string type. MessagePack had a fixed protocol spec before the IETF did.

The continued pursuit of the IETF fork was ostensibly been motivated by a disapproval of the development process and in particular a sense that the IETF process was superior. However, it seems to me that the IETF process was abused by CBOR's proponent, who just wanted things their own way. I have seen claims by IETF proponents that the open decisionmaking system inherently produces superior results. However, in this case the IETF process produced a bad specification. To the extent that other IETF contributors had influence over the ultimate CBOR RFC, I don't think they significantly improved it. CBOR has been described as MessagePack bikeshedded by the IETF. That would have been bad enough, but I think it's worse than that. To a large extent CBOR is one person's NIH-induced bad design rubber stamped by the IETF. CBOR's problems are not simply matters of taste: it's significantly overcomplicated.

One lesson for the rest of us is that although being the upstream and nominally in charge of a project seems to give us a lot of power, it's wise to listen carefully to one's users and downstreams. Once people are annoyed enough to fork, the fork will have a life of its own.

Another lesson is that many of us should be much warier of the supposed moral authority of the IETF. Many IETF standards are awful (Oauth 2 [12]; IKE; DNSSEC; the list goes on). Sometimes (especially when network adoption effects are weak, as with MessagePack vs CBOR) better results can be obtained from a smaller group, or even an individual, who simply need the thing for their own uses.

Finally, governance systems of public institutions like the IETF need to be robust in defending the interests of outsiders (and hence of society at large) against eloquent insiders who know how to work the process machinery. Any institution which nominally serves the public good faces a constant risk of devolving into self-servingness. This risk gets worse the more powerful and respected the institution becomes.

References

  1. #13: First-class string type in serialization specification (MessagePack issue tracker, June 2010 - August 2013)
  2. #121: Msgpack can't differentiate between raw binary data and text strings (MessagePack issue tracker, November 2012 - February 2013)
  3. draft-bormann-apparea-bpack-00: The binarypack JSON-like representation format (IETF Internet-Draft, October 2012)
  4. #129: MessagePack should be developed in an open process (MessagePack issue tracker, February 2013 - March 2013)
  5. Re: JSON mailing list and BoF (IETF apps-discuss mailing list message from Carsten Bormann, 18 February 2013)
  6. #128: Discussions on the upcoming MessagePack spec that adds the string type to the protocol (MessagePack issue tracker, February 2013 - August 2013)
  7. draft-bormann-apparea-bpack-01: The binarypack JSON-like representation format (IETF Internet-Draft, February 2013)
  8. draft-bormann-cbor: Concise Binary Object Representation (CBOR) (IETF Internet-Drafts, May 2013 - September 2013)
  9. RFC 7049: Concise Binary Object Representation (CBOR) (October 2013)
  10. "MessagePack should be replaced with [CBOR] everywhere ..." (floatboth on Hacker News, 8th April 2017)
  11. Discussion with very useful set of history links (camgunz on Hacker News, 9th April 2017)
  12. OAuth 2.0 and the Road to Hell (Eran Hammer, blog posting from 2021, via Wayback Machine)
  13. Re: [apps-discuss] [Json] msgpack/binarypack (Re: JSON mailing list and BoF) (IETF list message from Sadyuki Furuhashi, 4th March 2013)
  14. "no apologies for complaining about this farce" (IETF list message from Phillip Hallam-Baker, 15th August 2013) Edited 2020-07-14 18:55 to fix a minor formatting issue



comments

Markus Koschany: My Free Software Activities in June 2020

14 July, 2020 - 20:31

Welcome to gambaru.de. Here is my monthly report (+ the first week in July) that covers what I have been doing for Debian. If you’re interested in Java, Games and LTS topics, this might be interesting for you.

Debian Games Short news
  • The last month saw a new upstream release of Minetest (version 5.3.), a multi-player sandbox game similar to Minecraft. A backport to buster-backports will follow shortly.
  • Asher Gordon helped release a new version of Berusky 2, a sokoban like logic game but in 3D. The game received several improvements including bug fixes, code polishing and a new way to access the data files. Previously those files were all packed in a special container format but now they can be accessed directly without someone having to rely on some sort of unarchiver. I have uploaded this version as 0.12-1 to Debian unstable.
  • I tested an upstream patch for empire to address the build failure with GCC 10. This one is a better solution than the currently implemented workaround and I expect it to be included in the next upstream release.
  • I fixed two FTBFS in simutrans-pak64 and simutrans-pak128.britain, two addon packages for the simulation game simutrans.
Debian Java
  • New upstream versions this month: hsqldb, libpdfbox2-java, jackson-jr, jackson-dataformat-xml and jackson-databind. The latter upload addressed several security vulnerabilites which have become rather minor because upstream has enabled safe default typing by default now. Nevertheless I have prepared a buster-security update as well which is already available in buster-proposed-updates.
Misc
  • I packaged new versions of wabt, privacybadger and binaryen and applied another upstream patch for xarchiver to address the incomplete fix for Debian bug #959914, to better handle encrypted multi-volume 7zip archives.
  • By popular request I uploaded imlib2 version 1.6 to buster-backports because the image library supports the webp format now.
Debian LTS

This was my 52. month as a paid contributor and I have been paid to work 60 hours on Debian LTS, a project started by Raphaël Hertzog. In that time I did the following:

  • DLA-2278-1. Issued a security update for squid3 fixing 19 CVE.
  • DLA-2279-1. Issued a security update for tomcat8 fixing 2 CVE.
  • Prepared and uploaded a stretch-pu update for jackson-databind fixing 20 CVE. (#964727)
  • Synced the proftpd-dfsg version from Jessie with Stretch to address a memory leak which leads to a denial-of-service and correct the version number to make seemless updates work.
  • Prepared the security update for imagemagick triaging and/or fixing 76 CVE.
  • Worked on updating the database about embedded code copies to determine how packages are affected by security vulnerabilities in embedded code copies. This included a) compiling a list of important packages which are regular affected by CVE, b) investigating if embedded code copies are present, c) determining the possible impact of a security vulnerability in those embedded code copies, d) writing a script that automates printing those findings on demand.

Thanks for reading and see you next time.

Jonathan Dowland: Lockdown music

14 July, 2020 - 20:17

Last Christmas, to make room for a tree, I dis-assembled my hifi unit and (temporarily, I thought) lugged my hi-fi and records up to the study.

I had been thinking about expanding the amount of storage I had for hifi and vinyl, perhaps moving from 2x2 storage cubes to 2x3, although Ikea don't make a 2x6 version of their stalwart vinyl storage line, the Kallax. I had begun exploring other options, both at Ikea and other places. Meanwhile, I re-purposed my old Expedit unit as storage for my daughter's Sylvanian Families.

Under Lockdown, I've spent a lot more time in my study, and so I've set up the hifi there. It turns out I have a lot more opportunity to enjoy the records up here, during work, and I've begun to explore some things which I haven't listened to in a long time, or possibly ever. I thought I'd start keeping track of some of them.

Power, Corruption and Lies is not something rarely listened to. It's steadily become my favourite New Order album. When I came across this copy (Factory, 1983), it was in pristine condition, but it now bears witness to my (mostly careful) use. There's now a scratch somewhere towards the end of the first track Age of Consent which causes my turntable to loop. By some good fortune the looping point is perfectly aligned to a bar. I don't always notice it straight away. This record rarely makes it back down from the turntable to where it's supposed to live.

Bits from Debian: Let's celebrate DebianDay 2020 around the world

14 July, 2020 - 19:00

We encourage our community to celebrate around the world the 27th Debian anniversary with organized [DebianDay][1] events. This year due to the COVID-19 pandemic we cannot organize in-person events, so we ask instead that contributors, developers, teams, groups, maintainers, and users promote The Debian Project and Debian activities online on August 16th (and/or 15th).

Communities can organize a full schedule of online activities throughout the day. These activities can include talks, workshops, active participation with contributions such as translations assistance or editing, debates, BoFs, and all of this in your local language using tools such as [Jitsi][2] for capturing audio and video from presenters for later streaming to YouTube.

If you are not aware of any local community organizing a full event or you don't want to join one, you can solo design your own activity using [OBS][3] and stream it to YouTube. You can watch an OBS tutorial [here][4].

Don't forget to record your activity as it will be a nice idea to upload it to [Peertube][5] later.

Please add your event/activity on the [DebianDay wiki page][6] and let us know about and advertise it on [Debian micronews][7]. To share it, you have several options:

  • Follow the steps listed [here][8] for Debian Developers.
  • Contact us using IRC in channel #debian-publicity on the OFTC network, and ask us there.
  • Send a mail to debian-publicity@lists.debian.org and ask for your item to be included in micronews. This is a publicly archived list.

PS: DebConf20 online is coming! It will be held from August 23rd to 29th, 2020. [Registration is already open][9].

[1] https://wiki.debian.org/DebianDay [2] https://meet.jit.si [3] https://obsproject.com [4] https://peertube.debian.social/videos/watch/7f41c0e7-66cc-4234-b929-6b3219d95c14 [5] https://peertube.debian.social [6] https://wiki.debian.org/DebianDay/2020 [7] https://micronews.debian.org [8] https://micronews.debian.org/pages/contribute.html [9] https://debconf20.debconf.org/news/2020-07-12-registration-is-open/

Russell Coker: Debian PPC64EL Emulation

14 July, 2020 - 10:29

In my post on Debian S390X Emulation [1] I mentioned having problems booting a Debian PPC64EL kernel under QEMU. Giovanni commented that they had PPC64EL working and gave a link to their site with Debian QEMU images for various architectures [2]. I tried their image which worked then tried mine again which also worked – it seemed that a recent update in Debian/Unstable fixed the bug that made QEMU not work with the PPC64EL kernel.

Here are the instructions on how to do it.

First you need to create a filesystem in an an image file with commands like the following:

truncate -s 4g /vmstore/ppc
mkfs.ext4 /vmstore/ppc
mount -o loop /vmstore/ppc /mnt/tmp

Then visit the Debian Netinst page [3] to download the PPC64EL net install ISO. Then loopback mount it somewhere convenient like /mnt/tmp2.

The package qemu-system-ppc has the program for emulating a PPC64LE system, the qemu-user-static package has the program for emulating PPC64LE for a single program (IE a statically linked program or a chroot environment), you need this to run debootstrap. The following commands should be most of what you need.

apt install qemu-system-ppc qemu-user-static

update-binfmts --display

# qemu ppc64 needs exec stack to solve "Could not allocate dynamic translator buffer"
# so enable that on SE Linux systems
setsebool -P allow_execstack 1

debootstrap --foreign --arch=ppc64el --no-check-gpg buster /mnt/tmp file:///mnt/tmp2
chroot /mnt/tmp /debootstrap/debootstrap --second-stage

cat << END > /mnt/tmp/etc/apt/sources.list
deb http://mirror.internode.on.net/pub/debian/ buster main
deb http://security.debian.org/ buster/updates main
END
echo "APT::Install-Recommends False;" > /mnt/tmp/etc/apt/apt.conf

echo ppc64 > /mnt/tmp/etc/hostname

# /usr/bin/awk: error while loading shared libraries: cannot restore segment prot after reloc: Permission denied
# only needed for chroot
setsebool allow_execmod 1

chroot /mnt/tmp apt update
# why aren't they in the default install?
chroot /mnt/tmp apt install perl dialog
chroot /mnt/tmp apt dist-upgrade
chroot /mnt/tmp apt install bash-completion locales man-db openssh-server build-essential systemd-sysv ifupdown vim ca-certificates gnupg
# install kernel last because systemd install rebuilds initrd
chroot /mnt/tmp apt install linux-image-ppc64el
chroot /mnt/tmp dpkg-reconfigure locales
chroot /mnt/tmp passwd

cat << END > /mnt/tmp/etc/fstab
/dev/vda / ext4 noatime 0 0
#/dev/vdb none swap defaults 0 0
END

mkdir /mnt/tmp/root/.ssh
chmod 700 /mnt/tmp/root/.ssh
cp ~/.ssh/id_rsa.pub /mnt/tmp/root/.ssh/authorized_keys
chmod 600 /mnt/tmp/root/.ssh/authorized_keys

rm /mnt/tmp/vmlinux* /mnt/tmp/initrd*
mkdir /boot/ppc64
cp /mnt/tmp/boot/[vi]* /boot/ppc64

# clean up
umount /mnt/tmp
umount /mnt/tmp2

# setcap binary for starting bridged networking
setcap cap_net_admin+ep /usr/lib/qemu/qemu-bridge-helper

# afterwards set the access on /etc/qemu/bridge.conf so it can only
# be read by the user/group permitted to start qemu/kvm
echo "allow all" > /etc/qemu/bridge.conf

Here is an example script for starting kvm. It can be run by any user that can read /etc/qemu/bridge.conf.

#!/bin/bash
set -e

KERN="kernel /boot/ppc64/vmlinux-4.19.0-9-powerpc64le -initrd /boot/ppc64/initrd.img-4.19.0-9-powerpc64le"

# single network device, can have multiple
NET="-device e1000,netdev=net0,mac=02:02:00:00:01:04 -netdev tap,id=net0,helper=/usr/lib/qemu/qemu-bridge-helper"

# random number generator for fast start of sshd etc
RNG="-object rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0"

# I have lockdown because it does no harm now and is good for future kernels
# I enable SE Linux everywhere
KERNCMD="net.ifnames=0 noresume security=selinux root=/dev/vda ro lockdown=confidentiality"

kvm -drive format=raw,file=/vmstore/ppc64,if=virtio $RNG -nographic -m 1024 -smp 2 $KERN -curses -append "$KERNCMD" $NET

Related posts:

  1. Debian S390X Emulation I decided to setup some virtual machines for different architectures....
  2. installing Xen domU on Debian Etch I have just been installing a Xen domU on Debian...
  3. Installing a Red Hat based DomU on a Debian Dom0 The first step is to copy /images/xen/vmlinuz and /images/xen/initrd.img from...

Antoine Beaupré: Not recommending Purism

14 July, 2020 - 05:15

This is just a quick note to mention that I have updated my hardware documentation on the Librem 13v4 laptop. It has unfortunately turned into a rather lengthy (and ranty) piece about Purism. Let's just say that waiting weeks for your replacement laptop (yes, it died again) does wonders for creativity. To quote the full review:

TL;DR: I recommend people avoid the Purism brand and products. I find they have questionable politics, operate in a "libre-washing" fashion, and produce unreliable hardware. Will not buy again.

People who have read the article might want to jump directly to the new sections:

I have also added the minor section of the missing mic jack.

I realize that some folks (particularly at Debian) might still work at Purism, and that this article might be demoralizing for their work. If that is the case, I am sorry this article triggered you in any way and I hope this can act as a disclaimer. But I feel it is my duty to document the issues I am going through, as a user, and to call bullshit when I see it (let's face it, the anti-interdiction stuff and the Purism 5 crowd-funding campaign were total bullshit).

I also understand that the pandemic makes life hard for everyone, and probably makes a bad situation at Purism worse. But those problems existed before the pandemic happened. They were issues I had identified in 2019 and that I simply never got around to document.

I wish that people wishing to support the free software movement would spend their energy towards organisations that actually do honest work in that direction, like System76 and Pine64. And if you're going to go crazy with an experimental free hardware design, why not go retro with the MNT Reform.

In the meantime, if you're looking for a phone, I recommend you give the Fairphone a fair chance. It really is a "fair" (as in, not the best, but okay) phone that you can moderately liberate, and it actually frigging works. See also my hardware review of the FP2.

Bits from Debian: Debian Long Term Support (LTS) users and contributors survey

13 July, 2020 - 19:00

On July 18th Stretch LTS starts, offering two more years of security support to the Debian Stretch release. Stretch LTS will be the fourth iteration of LTS, following Squeeze LTS which started in 2014, Wheezy LTS in 2016 and Jessie LTS in 2018.

However, for the first time, we have prepared a small survey about our users and contributors, who they are and why they are using LTS.

Filling out the survey should take less than 10 minutes. We would really appreciate if you could participate in the survey online!

In two weeks (July 27th 2020) we will close the survey, so please don't hesitate and participate now! After that, there will be a followup email with the results.

More information about Debian LTS is available at https://wiki.debian.org/LTS, including generic contact information.

Click here to fill out the survey now!

Antoine Beaupré: On contact tracing

13 July, 2020 - 06:58

I have strong doubts about the efficiency of any tracing app of the sort, and even less in the context where it is unlikely that a majority of the population will use it.

There's also the problem that this app would need to work on Apple phones, or be incompatible with them, and cause significant "fracture" between those who have access to technology, and those who haven't. See this text for more details.

Such an app would be a security and privacy liability at no benefit to public health. There are better options, see for this research on hardware tokens. But I doubt any contact tracing app or hardware will actually work anyways.

I am a computer engineer with more than 20 years of experience in the domain, and I have been following this question closely.

Please don't do this.

I wrote the above in a response to the Québec government's survey about a possible tracing app.

Pour une raison que je m'explique mal, le sondage m'été envoyé en anglais, et j'ai donc écrit ma réponse dans la langue de Shakespeare au lieu de celle de molière... Je serai heureux de fournir une traduction française à ceux ou celles qui en ont besoin...

Enrico Zini: Police brutality links

13 July, 2020 - 05:00
Confessions of a Former Bastard Cop police politics archive.org 2020-07-13 I was a police officer for nearly ten years and I was a bastard. We all were. I was a police chief stopped by my own officer. After Floyd, we need change at all levels. police politics archive.org 2020-07-13 Hi White People! … I want to tell you something about my experience with white progressive backlash police politics archive.org 2020-07-13 We've detected that JavaScript is disabled in your browser. Would you like to proceed to legacy Twitter? Police: Last Week Tonight with John Oliver police politics archive.org 2020-07-13 As nationwide protests over the deaths of George Floyd and Breonna Taylor are met with police brutality, John Oliver discusses how the histories of policing ... Morte di Stefano Cucchi - Wikipedia italy police politics archive.org 2020-07-13 La morte di Stefano Cucchi avvenne a Roma il 22 ottobre 2009 mentre il giovane era sottoposto a custodia cautelare. Le cause della morte e le responsabilità sono oggetto di procedimenti giudiziari che hanno coinvolto da un lato i medici dell'ospedale Pertini,[1][2][3][4] dall'altro continuano a coinvolgere, a vario titolo, più militari dell’Arma dei Carabinieri[5][6]. Il caso ha attirato l'attenzione dell'opinione pubblica a seguito della pubblicazione delle foto dell'autopsia, poi riprese da agenzie di stampa, giornali e telegiornali italiani[7]. La vicenda ha ispirato, altresì, documentari e lungometraggi cinematografici.[8][9][10] Morte di Giuseppe Uva - Wikipedia italy police politics archive.org 2020-07-13 La morte di Giuseppe Uva avvenne il 14 giugno 2008 dopo che, nella notte tra il 13 e il 14 giugno, era stato fermato ubriaco da due carabinieri che lo portarono in caserma, dalla quale venne poi trasferito, per un trattamento sanitario obbligatorio, nell'ospedale di Varese, dove morì la mattina successiva per arresto cardiaco. Secondo la tesi dell'accusa, la morte fu causata dalla costrizione fisica subita durante l'arresto e dalle successive violenze e torture che ha subito in caserma. Il processo contro i due carabinieri che eseguirono l'arresto e contro altri sei agenti di polizia ha assolto gli imputati dalle accuse di omicidio preterintenzionale e sequestro di persona[1][2][3][4]. Alla vicenda è dedicato il documentario Viva la sposa di Ascanio Celestini[1][5]. Caso Aldrovandi - Wikipedia italy police politics archive.org 2020-07-13 Il caso Aldrovandi è la vicenda giudiziaria causata dall'uccisione di Federico Aldrovandi, uno studente ferrarese, avvenuta il 25 settembre 2005 a seguito di un controllo di polizia.[1][2][3] I procedimenti giudiziari hanno condannato, il 6 luglio 2009, quattro poliziotti a 3 anni e 6 mesi di reclusione, per "eccesso colposo nell'uso legittimo delle armi";[1][4] il 21 giugno 2012 la Corte di cassazione ha confermato la condanna.[1] All'inchiesta per stabilire la cause della morte ne sono seguite altre per presunti depistaggi e per le querele fra le parti interessate.[1] Il caso è stato oggetto di grande attenzione mediatica e ha ispirato un documentario, È stato morto un ragazzo.[1][5] Federico Aldrovandi - Wikipedia italy police politics archive.org 2020-07-13 Federico Aldrovandi (17 July 1987 in Ferrara – 25 September 2005 in Ferrara) was an Italian student, who was killed by four policemen.[1] Aldrovandi, il film di Vendemmiati gratis on line – Articolo21 italy police politics archive.org 2020-07-13 24 Giugno 2020

Evgeni Golov: Using Ansible Molecule to test roles in monorepos

12 July, 2020 - 15:03

Ansible Molecule is a toolkit for testing Ansible roles. It allows for easy execution and verification of your roles and also manages the environment (container, VM, etc) in which those are executed.

In the Foreman project we have a collection of Ansible roles to setup Foreman instances called forklift. The roles vary from configuring Libvirt and Vagrant for our CI to deploying full fledged Foreman and Katello setups with Proxies and everything. The repository also contains a dynamic Vagrant file that can generate Foreman and Katello installations on all supported Debian, Ubuntu and CentOS platforms using the previously mentioned roles. This feature is super helpful when you need to debug something specific to an OS/version combination.

Up until recently, all those roles didn't have any tests. We would run ansible-lint on them, but that was it.

As I am planning to do some heavier work on some of the roles to enhance our upgrade testing, I decided to add some tests first. Using Molecule, of course.

Adding Molecule to an existing role is easy: molecule init scenario -r my-role-name will add all the necessary files/examples for you. It's left as an exercise to the reader how to actually test the role properly as this is not what this post is about.

Executing the tests with Molecule is also easy: molecule test. And there are also examples how to integrate the test execution with the common CI systems.

But what happens if you have more than one role in the repository? Molecule has support for monorepos, however that is rather limited: it will detect the role path correctly, so roles can depend on other roles from the same repository, but it won't find and execute tests for roles if you run it from the repository root. There is an undocumented way to set MOLECULE_GLOB so that Molecule would detect test scenarios in different paths, but I couldn't get it to work nicely for executing tests of multiple roles and upstream currently does not plan to implement this. Well, bash to the rescue!

for roledir in roles/*/molecule; do
    pushd $(dirname $roledir)
    molecule test
    popd
done

Add that to your CI and be happy! The CI will execute all available tests and you can still execute those for the role you're hacking on by just calling molecule test as you're used to.

However, we can do even better.

When you initialize a role with Molecule or add Molecule to an existing role, there are quite a lot of files added in the molecule directory plus an yamllint configuration in the role root. If you have many roles, you will notice that especially the molecule.yml and .yamllint files look very similar for each role.

It would be much nicer if we could keep those in a shared place.

Molecule supports a "base config": a configuration file that gets merged with the molecule.yml of your project. By default, that's ~/.config/molecule/config.yml, but Molecule will actually look for a .config/molecule/config.yml in two places: the root of the VCS repository and your HOME. And guess what? The one in the repository wins (that's not yet well documented). So by adding a .config/molecule/config.yml to the repository, we can place all shared configuration there and don't have to duplicate it in every role.

And that .yamllint file? We can also move that to the repository root and add the following to Molecule's (now shared) configuration:

lint: yamllint --config-file ${MOLECULE_PROJECT_DIRECTORY}/../../.yamllint --format parsable .

This will define the lint action as calling yamllint with the configuration stored in the repository root instead of the project directory, assuming you store your roles as roles/<rolename>/ in the repository.

And that's it. We now have a central place for our Molecule and yamllint configurations and only need to place role-specific data into the role directory.

Dirk Eddelbuettel: drat 0.1.7: New functionality

12 July, 2020 - 09:56

A new version of drat arrived on CRAN yesterday. Once again, this release is mostly the work of Felix Ernst who extended some work from the previous release, and added support for repository updates (outside of package insertion) and more.

drat stands for drat R Archive Template, and helps with easy-to-create and easy-to-use repositories for R packages. Since its inception in early 2015 it has found reasonably widespread adoption among R users because repositories with marked releases is the better way to distribute code.

As your mother told you: Friends don’t let friends install random git commit snapshots. Rolled-up releases it is. drat is easy to use, documented by five vignettes and just works.

The NEWS file summarises the release as follows:

Changes in drat version 0.1.7 (2020-07-10)
  • Functions insertPackages, archivePackages and prunePackages are now vectorised (Patrick Schratz and Felix Ernst in #93, #100).

  • The new functionality is supported by unit tests (Felix Ernst in #93, and #102 fixing #101).

  • Added new function updateRepo (Felix Ernst in #95, #97).

Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

If you like this or other open-source work I do, you can now sponsor me at GitHub. For the first year, GitHub will match your contributions.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Simon Quigley: Adventures in Writing

11 July, 2020 - 17:59

The Linux community is a fascinating and powerful space.

When I joined the Ubuntu project approximately five years ago, I (vaguely at the time) understood that there was a profound sense of community and passion everywhere that is difficult to find in other spaces. My involvement has increased, and so has my understanding. I had thought of starting a blog as a means of conveying the information that I stumbled across, but my writing skills were very crude and regrettable, being in my early teenage years.

I have finally decided to take the leap. In this blog, I would like to occasionally provide updates on my work, either through focused deep dives on a particular topic, or broad updates on low hanging fruit that has been eliminated. While the articles may be somewhat spontaneous, I decided that an initial post was in order to explain my goals. Feel free to subscribe for more detailed posts in the future, as there are many more to come.

Enrico Zini: Wait until a command opened a file

11 July, 2020 - 00:27

In my last post I wrote:

The sleep 0.3s is needed because xdg-open exits right after starting the program, and when invoked by mutt it means that mutt could delete the attachment before evince has a chance to open it. I had to use the same workaround for sensible-browser, since the same happens when a browser opens a document in an existing tab. I feel like writing some wrapper about all this that forks the viewer, then waits for an IN_OPEN event on its argument via inotify before exiting.

I wrote it: https://github.com/spanezz/waitused/

$ ./waitused --help
usage: waitused [-h] path ...

Run a command exiting only after it quits and a given file has been opened and
closed

positional arguments:
  path        file to monitor
  command     command to run

optional arguments:
  -h, --help  show this help message and exit

This works around situations like mutt deleting the temporary attachment file after run-mailcap is run, while run-mailcap runs a program that backgrounds before opening its input file.

Example
waitused file.pdf xdg-open file.pdf
waitused file.pdf run-mailcap file.pdf
Example ~/.mailcap entry
application/pdf; waitused %s xdg-open %s; test=test -n "$DISPLAY"

Pages

Creative Commons License ลิขสิทธิ์ของบทความเป็นของเจ้าของบทความแต่ละชิ้น
ผลงานนี้ ใช้สัญญาอนุญาตของครีเอทีฟคอมมอนส์แบบ แสดงที่มา-อนุญาตแบบเดียวกัน 3.0 ที่ยังไม่ได้ปรับแก้