Added rosdep init-mirror and rosdep update-mirror instructions.

[anchuanxu]

PR description

-Two rosdep directives were added to solve the problem of slow visit in http://raw.githubusercontent.com, China.
-The mirror source: https://mirrors.tuna.tsinghua.edu.cn/ros/ is used.

Introduction of new instructions:

• rosdep init-mirror;
• rosdep update-mirror。

[ZenithalHourlyRate]

Hi, is there any further update on this PR. Users of rosdep in China have experienced bad usability due to the network issue when accessing raw.githubusercontent, so this PR is wanted.

I'm a maintainer of TUNA mirror, and more specifically I mirrorred 20-default.list in our syncing script

One more thing to note is that now BFSU also supports this.

[cottsay]

Instructing this tool to use alternate rosdep db URLs has always been supported out of the box. I don't believe that adding additional verbs and a hard-coded list of mirrors to do so is necessary, desirable, or maintainable. Releasing a new version of this package if/when the mirror list changes isn't a good workflow.

This PR already requires users to change the commands they invoke. Why not simply do this instead of sudo rosdep init:
sudo curl -o /etc/ros/rosdep/sources.list.d/10-mirror.list https://mirrors.tuna.tsinghua.edu.cn/github-raw/ros/rosdistro/master/rosdep/sources.list.d/20-default.list

...and then use rosdep normally?

Another thing to note is that this PR doesn't instruct rosdistro to use a mirror, so you would still need access to GitHub in order to download the package list from the rosdistro index. As it happens, using an alternate URL for that is also supported out of the box using an environment variable: export ROSDISTRO_INDEX_URL=https://example.com/index-v4.yaml. It doesn't look like the mirrors you linked support that, so at best, this seems like a mitigation and will not solve a complete lack of connectivity to GitHub.

[ZenithalHourlyRate]

Thanks for this feedback! Knowing what ros people think on this issue would help us settle the problem for our users. If this is not the way things are done, we can try to solve it in another way.

Instructing this tool to use alternate rosdep db URLs has always been supported out of the box.

rosdep/src/rosdep2/main.py

Line 597 in 16e4b3a

data = download_default_sources_list()

After a quick digging I could not find a environment variable that are passed here (Maybe I missed some code, correct me if I'm wrong). May be some environment variable like ROSDEP_SOURCES_LIST_URL can be passed here? Or we can let our user skip rosdep init and instead setup like this

sudo mkdir -p /etc/ros/rosdep/sources.list.d/
sudo curl -o /etc/ros/rosdep/sources.list.d/10-mirror.list https://mirrors.tuna.tsinghua.edu.cn/github-raw/ros/rosdistro/master/rosdep/sources.list.d/20-default.list
rosdep update

I understand your new-verb/maintenance issue, so if it is not the way, we can instead use env var here.

It doesn't look like the mirrors you linked support that

We can add related infra for that. As I'm not a daily user of rosdep, it seems that mirroring a whole checkout of ros/rosdistro and letting the user use ROSDISTRO_INDEX_URL would solve the problem? Correct me if I missed anything.

One more thing to note is that our user can connect github.com most of the time but almost can not connect raw.githubusercontent .

So it seems that the takeaway is that changes needed are we mirror site recommends our user to use the above workaround, and we sync the whole ros/rosdistro.

If there is indeed something that rosdep needs to adopt, I think the ROSDEP_SOURCES_LIST_URL can be one.

I do not know if it works out of the box to get ros/rosdistro from a mirror site. I haven't read the REP now, is
https://github.com/ros/rosdistro/blob/912e14de5e1b6b0b239258ff917651be2e46b139/index-v4.yaml#L7
a relative path and further download can all be done from a base path from ROSDISTRO_INDEX_URL?

Again thanks for your information on these env vars, I think your information would help us develop a way to solve the problem. (I've seen tons of issue on the connectivity and if this is settled, our user would be more happy and all these issues can be solved)

[ZenithalHourlyRate]

a relative path and further download can all be done from a base path from ROSDISTRO_INDEX_URL

It works as intended.

So we have setup a mirror at https://mirrors.tuna.tsinghua.edu.cn/rosdistro/, see https://mirrors.tuna.tsinghua.edu.cn/help/rosdistro/ for usage

Could we add this to the wiki? I think with this mirror you can close some issue now.

Thank you again for pointing out the environment variable.

[anchuanxu]

Instructing this tool to use alternate rosdep db URLs has always been supported out of the box. I don't believe that adding additional verbs and a hard-coded list of mirrors to do so is necessary, desirable, or maintainable. Releasing a new version of this package if/when the mirror list changes isn't a good workflow.

This PR already requires users to change the commands they invoke. Why not simply do this instead of sudo rosdep init: sudo curl -o /etc/ros/rosdep/sources.list.d/10-mirror.list https://mirrors.tuna.tsinghua.edu.cn/github-raw/ros/rosdistro/master/rosdep/sources.list.d/20-default.list

...and then use rosdep normally?

Another thing to note is that this PR doesn't instruct rosdistro to use a mirror, so you would still need access to GitHub in order to download the package list from the rosdistro index. As it happens, using an alternate URL for that is also supported out of the box using an environment variable: export ROSDISTRO_INDEX_URL=https://example.com/index-v4.yaml. It doesn't look like the mirrors you linked support that, so at best, this seems like a mitigation and will not solve a complete lack of connectivity to GitHub.

thanks for your help!

[anchuanxu]

a relative path and further download can all be done from a base path from ROSDISTRO_INDEX_URL

It works as intended.

So we have setup a mirror at https://mirrors.tuna.tsinghua.edu.cn/rosdistro/, see https://mirrors.tuna.tsinghua.edu.cn/help/rosdistro/ for usage

Could we add this to the wiki? I think with this mirror you can close some issue now.

Thank you again for pointing out the environment variable.

厉害了我的哥，这个方式来解决这个问题是恰当的。赞！

[cottsay]

Could we add this to the wiki?

That seems like a good idea to me. Please do so.

[ZenithalHourlyRate]

Could we add this to the wiki?

That seems like a good idea to me. Please do so.

Added in wiki (http://wiki.ros.org/rosdep). PR on ROS2 doc in progress (ros2/ros2_documentation#2842)

I assume this PR could be closed now?