There and back again

生も死もひとつのブランコの分岐点においてゆらゆらそのゆれをほほえんで見ているしかないのですね。

TeX Services at texlive.info

by ·

I have been working over the last weeks to provide four more services for the TeX (Live) community: an archive of TeX Live’s network installation directory tlnet, a git repository of CTAN, a mirror of the TeX Live historic archives, and a new tlpretest mirror. In addition to the services that have already been provided before on my server, this makes a considerable list, and I thought it is a good idea to summarize all of the services.

Overview of the services

New services added recently are marked with an asterisk (*) at the end.

For the git services, anonymous checkouts are supported. If a developer wants to have push rights, please contact me.

tlnet archive

TeX Live is distributed via the CTAN network in CTAN/systems/texlive/tlnet. The packages there are updated on a daily basis according to updates on CTAN that make it into the TeX Live repository. This has created some problems for distributions requiring specific versions, as well as problems with rollbacks in case of buggy packages.

Starting with 2019/08/30, for each day rsync backups of the tlnet directory are done, and they are available at https://www.texlive.info/tlnet-archive/YYYY/MM/DD/tlnet.

CTAN git repository

The second big item is putting CTAN into a git repository. In a perfect world I could get git commits for each single package update, but that would need a lot of collaboration with the CTAN Team (and maybe this will happen in the future), but for now there is one rsync of the CTAN a day committed after the sync.

Considering the total size of CTAN (currently around 40G), we decided to ignore file types that provide no useful information when put into git, mostly large binary files. The concrete list is tar, zip, pkg, cab, jar, dmg, rpm, deb, tgz, iso, exe, cab, as well as files containing one of these extensions (that means that files foobar.iso.gz will be ignored, too). This allows to keep the size of the .git directory for now at something reasonable amount (a few Gb).

We will see how the git repository grows over time, and whether we can support this on a long term time range.

While we exclude the above files from being recorded in the git repository, the actual CTAN directory is complete and contains all files, meaning that rsync checkout contains everything.

Access to these services is provided as follows:

TeX Live historic archives

The TeX Live historic archives hierarchy contains various items of interest in TeX history, from individual files to entire systems. See the article by Ulrik Vieth at https://tug.org/TUGboat/tb29-1/tb91vieth.pdf for an overview.

We provide a mirror available via rsync://texlive.info/historic/.

tlpretest mirror

During preparation of a new TeX Live release (the pretest phase) we are distributing preliminary builds via a few tlpretest mirrors. The current server will provide access to tlpretest, too:

TeX Live svn/git mirror

Since I prefer to work with git, and developing new features with git on separate branches is so much more convenient than working with subversion, I am running a git-svn mirror of the whole TeX Live subversion repository. This repo is updated every 15min with the latest changes. There are also git branches matching the subversion branches, and some dev/ branches where I am working on new features. The git repository carries, similar to the subversion, the full history back to our switch from Perforce to Subversion in 2005.This repository is quite big, so don’t do a casual checkout (checked out size currently close to 40Gb):

TeX Live contrib

The TeX Live Contrib repository is a companion to the core TeX Live (tlnet) distribution in much the same way as Debian’s non-free tree is a companion to the normal distribution. The goal is not to replace TeX Live: packages that could go into TeX Live itself should stay (or be added) there. The TeX Live Contrib is simply trying to fill in a gap in the current distribution system by providing ready made packages for software that is not distributed in TeX Live proper due to license reasons, support for non-free software, etc.:

TeX Live GnuPG

Starting with release 2016, TeX Live provides facilities to verify authenticity of the TeX Live database using cryptographic signatures. For this to work out, a working GnuPG program needs to be available. In particular, either gpg (version 1) or gpg2 (version 2). To ease adoption of verification, this repository provides a TeX Live package tlgpg that ships GnuPG binaries for Windows and MacOS (universal and x86_64). On other systems we expect GnuPG to be installed.

Supporting these services

We will try to keep this service up and running as long as server space, connectivity, and bandwidth allows. If you find them useful, I happily accept donations via PayPal or Patreon to support the server as well as my time and energy!

Tags:

6 Responses

  1. riccardoporreca says:
    2019/11/13 at 07:50

    Thanks for the great work around TeX Live and the great texlive.info!

    I have a rather generic question about TeX Live: If I look at e.g. at the (frozen) TeXLive 2018 release in the tlnet archive (https://www.texlive.info/tlnet-archive/2019/02/27/tlnet/), I see all the expected versions (aligned with the git/svn repo at that date) of scripts under https://www.texlive.info/tlnet-archive/2019/02/27/tlnet/tlpkg/. Hoever, this is not the case for the same scripts in the texlive.infra package under https://www.texlive.info/tlnet-archive/2019/02/27/tlnet/archive/, which contains older versions.
    How does the build of texlive.infra works? Is it supposed to be aligned with the tlnet/tlpkg?

    To give you some context about where I am coming from, we are installing TeX Live (via TinyTeX) using https://www.texlive.info/tlnet-archive/2019/02/27/tlnet/ as repo, inside a Docker container for the Rocker project (https://github.com/rocker-org/rocker-versioned/pull/171#issuecomment-552671704). I was assuming this TeX Live repo would also pick the fix mentioned in https://github.com/yihui/tinytex/issues/77#issuecomment-466584510 and committed in https://git.texlive.info/texlive/commit/Master/tlpkg/TeXLive/TLPDB.pm?id=f5806adff881d4de9828bfb267d6995c79566d7a, which however is not reflected in the version of tlmgr installed as part of texlive.infra

    Reply
    • Norbert Preining says:
      2019/11/13 at 10:13

      Hi Riccardo
      thanks for your comment! I am not exactly sure what you mean with “under …/tlnet/tlpkg”, but my guess is that you are surprised that the versions of the Perl modules in tlnet/tlpkg/TeXLive are newer than the ones that are at the end installed on the system, is this correct?

      If yes, the explanation is as follows: The texlive.infra and some other packages are considered “critical” packages and are **not** automatically updated on CTAN/tlnet, but only in the so called tlcritical (see end of https://www.tug.org/texlive/tlmgr.html). BUT we do not care for using the current (most recent) Perl modules (and installer files) for the installer.

      The reason is simple: Shipping a broken texlive.infra to users could seriously hinder them to do any further updates, while shipping a broken installer only hinders users to install TeX Live for one day, until we fix the installer.

      That is, the version that is installed in the final system, and that is also the version registered in the tlnet/tlpkg/texlive.tlpdb, might carry a bit older files than the ones that are present in the installer and tlnet/tlpkg/TeXLive.

      I hope that makes it clear.

      In fact, now that you mentioned it, we should definitely push out tlcritical to tlnet before doing the last build and freeze, that might be indeed a good idea. But not necessarily, since sometimes we work already on changes for the next release which cannot be pushed out.

      Reply
      • riccardoporreca says:
        2019/11/14 at 04:22

        Hi Norbert, Thanks a lot for the prompt reply and the useful insights, you nailed exactly what I meant, which was indeed related to what I now know as the “Perl modules” being newer for the installer than in texlive.infra, and the process you describe makes indeed a lot of sense.

        About “we should definitely push out tlcritical to tlnet before doing the last build and freeze, that might be indeed a good idea”: that would be cool, indeed if not too much of a pain in case changes related to the next release are in scope.

        I guess we have to live with what is currently in texlive.infra, at least for the frozen TeX Live 2018 in the tlnet archive. I was then wondering if the following approach could be sensible: What if we overwrite the older Perl modules installed as part of texlive.infra with the more recent version used by the installer?
        I have actually tried this (on Debian/Ubuntu) already before commenting yesterday, by replacing tlpkg/TeXLive in the installation folder of TeX Live with the tlpkg/TeXLive inside https://www.texlive.info/tlnet-archive/2019/02/27/tlnet/install-tl-unx.tar.gz.
        This seems to work, but I am happy to get your advise about whether this is a strongly discouraged or whether this could cause any problem (I am not a Perl expert).

        Thanks a lot!

        Reply
        • Norbert Preining says:
          2019/11/14 at 10:16

          In this case, where only TLPDB.pm is changed, this can be done, feel free! Just as a warning, it is not always possible: If there are additional changes in tlmgr.pl pending, then this one would need to be obtained separately, which might be a problem (since it is not included in the installer).

          Reply
          • riccardoporreca says:
            2019/11/15 at 05:42

            Hi Robert, we will then pursue this approach for our case.

            Thanks again for all your help!

  1. Links 12/9/2019: Manjaro 18.1 and KaOS 2019.09 Releases | Techrights
    2019/09/13

    […] Norbert Preining: TeX Services at texlive.info […]

Leave a Reply

Your email address will not be published. Required fields are marked *