Updates on the Github to Codeberg migration

As we announced yesterday, we’ll be migrating from Github to Codeberg as our git host of choice.

Codeberg has a built-in migration tool that just works for most repositories, but alas it seems our repository is a bit too large. It’s not even Codeberg’s fault for we hit the Github rate limits in the middle of the migration.

I’ve tried a few different ways, including letting it run overnight to see if it would land on its feet somehow, but in the end it never worked.

So I’ve now written a script to migrate all git content, issues and pull request, and I am using that to handle the migration. It will almost certainly also go off the rails at some point, but my hope is that by scripting things myself, I will be able to restart from where it failed, rather than having to start all over again.

I will keep you posted on my progress, but it might take time because each time we hit the rate limits, we need to back off for a few hours to let them reset.

When the migration is complete, I will archive the Github repository to avoid confusion.
We will then also try to clean up the issues and pull requests that have accumulated as we were working on v4 (this too will take time).

3 Likes

Wow, I wouldn’t be surprised if GitHub makes it hard on purpose to migrate away from them. That feels like typical Microsoft behaviour.

Hope this works out without too much hassle.

Update on this: While I now have a script that will migrate things, there’s still some issues that we cannot get around:

  • Rate limiting: This means we have to go slow. Sometimes real slow. Because rate limiting kicks in when you create 5 whatever under 5 minutes. So 5 issues in 5 minutes will trigger it. But that’s not the end of it. One issue can have many comments, and each of those counts to, so it’s going to take time.
  • Pull requests: Github pull requests cannot be migrated to Codeberg pull requests as-is because they way they are stored is specific to Github. They will be created as issues instead.

What this means is that:

  • Pull requests will be recreated as issue in Codeberg. The main goal here is to preserve the content and discussion. When it comes to actually bringing the code into our codebase, I will just handle it with cross-repository merges (famous last words)
  • I will only migrate open issues and pull requests. I don’t know if there is much value in migrating closed issues (and pull requests) but if people feel particularly nostalgic about it, we can always deal with it later.

As all this will take a while, I will archive the repo on Github so that we can get out of this state of limbo and start pushing updates to the site as there’s some things that are broken and need attention.

2 Likes

I think just migrating open stuff is completely fine. The old stuff is still on GitHub if you need to look at it for some reason. And in the worst case, if you want to reopen an old GitHub issue, you could just manually recreate it and link the original if needed.

Not sure if you’re aware, but your script seems to create duplicate issues:

It did at first after I restarted it. Now it should no longer do that.
Seems I forgot to clean up that one, so I’ve removed it now. Thanks for pointing it out :folded_hands:

1 Like

The migration is/should be complete.

If there’s anything missing, let me know :slightly_smiling_face:

2 Likes

I think a few items are still missing. There were 170 issues and pull requests open on Github, but there are only 157 issues on codeberg.

E.g. Also sample sa, complete and paperless in design unit tests by HaasJona · Pull Request #7016 · freesewing/freesewing · GitHub was apparently not migrated.

Ah, I forgot to check the error logs. Seems some still failed due to rate limiting. Specifically:

  • 5563
  • 5637
  • 5742
  • 6177
  • 7016
  • 7158
  • 7180
  • 7213
  • 7242
  • 7327
  • 7333
  • 7357
  • 7389

I’ve taken care of them. Open issues count now matches (170).

1 Like

I haven’t yet signed up for a Codeberg account, but looking at the repository freesewing/freesewing: Freesewing's monorepo holding all our NPM packages, including our core library - Codeberg.org as a guest, I see that the Wiki wasn’t copied over. I’m not sure if Codeberg offers wiki functionality, though I see Forgejo does seem to have it.

(An alternative to using the wiki for unofficial documentation might be to use the forum instead.)

I also don’t see an equivalent to GitHub Discussions on Codeberg, but the forum seems like a better replacement for it anyway.

That’s my bad, I wrote the migration script and got lost in the issues/pull requests and then forgot about the wiki.

I’ll have a look and see if we can migrate it still, although I might need a few days because I have some other obligations this weekend.