The latest folder with redirects deleted by mike 2.0.0

[Vegebutcher]

Summary

After running a pipeline which builds a site using Material for MkDocs and mike, all redirects are gone (the general URL for docs doesn't work, e.g. https://my.docs.com). The latest folder is deleted by mike so the only way to open docs is to specify their version in the URL, e.g. https://my.docs.com/2.0. mike adds the latest file which contains the latest version number but it does not seem to be used anywhere.

Configuration and Logs

The git repository is private (GitLab), password-protected docs are deployed to Netlify from the pages branch.

• MkDocs 1.5.3 and mike 2.0.0
• CI scripts which call mike (first deploys docs without changing the latest alias):

  script:
    - mike deploy --update-aliases -b $PAGES_BRANCH -p $VERSION

  script:
    - mike deploy --update-aliases -p -b $PAGES_BRANCH $VERSION latest
    - mike set-default -b $PAGES_BRANCH -p latest

[jimporter]

Unfortunately, I can't provide much support on private repos or Gitlab deployments, much less both, so the most I can do is to give you a couple pointers on where to look:

1. How are you pushing your $PAGES_BRANCH to the web server? Specifically, are you sure that when you push this, your latest directory is a proper symlink? (On Windows, when you check out a symlink from Git, it defaults to a text file with the symlink's target as the contents.)
2. After pushing to your server, are you sure that the server itself properly handles symlinks?
3. If the above don't work for whatever reason, see the alias_type option. Depending on the behavior you'd like, you can use redirect (which makes a simple HTML redirect for each aliased page) or copy (which makes a duplicate of your entire site for the alias).

[Vegebutcher]

For now, I changed the requirements to stick to the good old mike 1.1.2 and ran the job again. And voila, the latest folder was re-created! So, it's something wrong with the latest version of mike, at least in my case.

Regarding your reply:

1. The pages branch is tracked by Netlify. This branch contains HTML files generated by mkdocs+mike. Whenever a new commit appears there, Netlify starts its job and builds and deploys the site. With the previous version of mike, the latest folder was created on this branch and it contained redirects to the latest versions of html files. mike 2.0.0 simply deleted this folder giving nothing in return but the latest file with just a version number in it. I don't know if it was intended or not, this is why I'm reporting an issue.
2. Everything worked until today when the CI in GL used mike 2.0.0. So yes, I guess the server properly handles symlinks.
3. I'll try these options next week. I had my Friday evening ruined trying to fix something that used to work…

[jimporter]

1. The pages branch is tracked by Netlify. This branch contains HTML files generated by mkdocs+mike. Whenever a new commit appears there, Netlify starts its job and builds and deploys the site. With the previous version of mike, the latest folder was created on this branch and it contained redirects to the latest versions of html files. mike 2.0.0 simply deleted this folder giving nothing in return but the latest file with just a version number in it. I don't know if it was intended or not, this is why I'm reporting an issue.

Yes, that's intended because aliases (e.g. latest) are now symlinks in your docs branch by default. It would seem that Netlify isn't smart enough to handle symlinks from Git. Perhaps this is the bug? I don't know anything about Netlify.

2. Everything worked until today when the CI in GL used mike 2.0.0. So yes, I guess the server properly handles symlinks.

Doubtful, since mike 1.x didn't use symlinks at all. That's a new feature in 2.0.

3. I'll try these options next week. I had my Friday evening ruined trying to fix something that used to work…

Sorry about that. However, 2.0.0 was published as a major version update to prevent exactly this sort of problem: by pinning your deps (at least to the current major version), you can avoid silent updates that contain backwards-incompatible changes. I've been careful with mike to avoid any breaking changes when publishing minor/patch-level releases.

[claudio-alvaro-wbgames]

I can confirm that this issue with mike 2.0 is also occurring on Github Enterprise Server 3.8. Symbolic links are not working. I checked the latest version docs (3.10) and in the release notes:

As a security measure, GitHub Pages does not build sites that contain symbolic links except when using custom GitHub Actions workflows. This change strengthens GitHub Pages's symbolic link detection.

https://docs.github.com/en/[email protected]/admin/release-notes

An excerpt of our workflow:

      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Publish docs
        run: |
          git config user.name github-workflow-bot
          git config user.email [email protected]
          git fetch origin
          mike deploy --push --update-aliases ${{ github.ref_name }} latest
          mike set-default --push latest

[jimporter]

Hmm. On the one hand, mike is primarily built to support Github Pages, so if it doesn't work by default on some Github instances, maybe symlinks should be opt-in. On the other hand, it might be more accurate to say that mike is primarily built to support github.com, so having it be opt-in would result in a worse behavior for the tool's main audience.

As far as I can tell, Github Enterprise is at least kind enough to report an error in this case (not that I've tried to be sure).

[claudio-alvaro-wbgames]

As suggested, I've resolved the issue by adding the alias-type=redirect option to the deploy command.

mike deploy --push --update-aliases ${{ github.ref_name }} latest --alias-type=redirect

[Sispheor]

We were also encountering the error Site contained a symlink that should be dereferenced: /latest.
With mike 1.1.2 the workaround was to add --no-redirect. E.G:

mike deploy --push --update-aliases 2.3.0 latest --remote upstream --no-redirect

[jimporter]

We were also encountering the error Site contained a symlink that should be dereferenced: /latest. With mike 1.1.2 the workaround was to add --no-redirect.

Are you sure about that? Version 1.1.2 doesn't use symlinks at all. It's a new feature in 2.0. (The "redirects" are small HTML pages with some HTML/JS redirect code, so the redirection happens client-side.)

[Sispheor]

mike --version                                                                                
mike 1.1.2

no-redirect flag:

mike deploy --help
usage: mike deploy [-h] [-t TITLE] [-u] [--no-redirect] [-T TEMPLATE] [-F FILE] [-r REMOTE] [-b BRANCH] [-m MESSAGE] [-p] [-f] [--prefix PATH] [--rebase | --ignore] VERSION [ALIAS ...]

Build the current documentation and deploy it to the specified version (and aliases, if any) on the target branch.

positional arguments:
  VERSION               version (directory) to deploy this build to
  ALIAS                 alias for this build (e.g. "latest")

options:
  -h, --help            show this help message and exit
  -t TITLE, --title TITLE
                        short descriptive title for this version
  -u, --update-aliases  update aliases pointing to other versions
  --no-redirect         make copies of docs for each alias
  -T TEMPLATE, --template TEMPLATE
                        the template file to use for redirects
  -F FILE, --config-file FILE
                        the MkDocs configuration file to use

git arguments:
  -r REMOTE, --remote REMOTE
                        origin to push to (default: origin)
  -b BRANCH, --branch BRANCH
                        branch to commit to (default: gh-pages)
  -m MESSAGE, --message MESSAGE
                        commit message
  -p, --push            push to {remote}/{branch} after commit
  -f, --force           force push when pushing
  --prefix PATH         subdirectory within {branch} where docs are located
  --rebase              rebase with remote
  --ignore              ignore remote status

[jimporter]

That flag exists in v1.1.2, but it's choosing between copying the entire site vs creating HTML redirect pages for each page of your docs. The symlink code is new in v2.0.0 (here's the commit), and represents a third option.

[Sispheor]

Well, it has fixed the prob concerning the recent Github behavior change with symlinks. So I just leave it there for other v1 users 🙂

[jimporter]

Looking at your commit log, I think that's because you used mike 2.0.0 for a single commit.

[jimporter]

@Sispheor Your repo's GH Actions log looks a bit different from what I'd expect. You might want to check the repo settings to make sure it's using the defaults. This is what I use, and it works just fine with symlinks:

[Sispheor]

Good catch! It's weird. Mike version is hard coded in the poetry descriptor. The ci should have kept V1. I'll investigate. Thanks for the heads-up!

[Vegebutcher]

As suggested, I've resolved the issue by adding the alias-type=redirect option to the deploy command.

mike deploy --push --update-aliases ${{ github.ref_name }} latest --alias-type=redirect

Thanks for this tip, @claudio-alvaro-wbgames, it worked! Just like @jimporter suggested in his first reply.

[Sispheor]

@jimporter Hi, Just to confirm that indeed my CI was installing mike without precising the version. And during the last publish it used the 2.0. Thanks again for the pointer !

[jimporter]

Tagging this as a 3.0 milestone change, since if we do adjust the default, it'll require a major version bump for semver compliance.

[ThinkChaos]

IMO using --alias-type=copy would be a better change than redirect since it's transparent for end-users.

Is there anything preventing making a 3.0 with just a fix for this, and pushing the rest of the milestone to a future 4.0?
It's a bit sad that the current default just doesn't work.

[jimporter]

@ThinkChaos I looked at the repo from your issue (0xERR0R/blocky#1329), and I don't think this issue has anything to do with your problem. Since you're hosting your docs on Github via https://0xerr0r.github.io/blocky/, the default mike configuration with symlinks should just work. I think you have a different issue. I'd suggest making a new issue for your problem and adding all the info you can to the new issue form.

(This issue occurs when people take the symlink-based aliases from mike and then try to copy their gh-pages branch to another hosting service like Netlify. Netlify doesn't understand symlinks - despite them being over 40 years old, grumble grumble - and so Netlify shows the contents of the symlink rather than dereferencing it.)

[ThinkChaos]

Thanks for the reply and checking that issue out! Turns out I got confused and the symlink seems to work fine.
I was a bit tired and there were a bunch of cases so I got some mixed up. So no bug to report :)