-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
More taylored instructions for iBridges #45
Merged
Merged
Changes from 13 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
73174de
More taylored instructions
chStaiger a3aaab3
Update ibridges.qmd
chStaiger f291827
Update docs/manuals/ibridges.qmd
chStaiger c57745e
Update docs/manuals/ibridges.qmd
chStaiger 3b267f6
Update docs/manuals/ibridges.qmd
chStaiger 174e05a
Update docs/manuals/ibridges.qmd
chStaiger 56ea991
Update docs/manuals/ibridges.qmd
chStaiger 084c3e0
Update docs/manuals/ibridges.qmd
chStaiger df3185c
Update docs/manuals/ibridges.qmd
chStaiger 65e8c31
Update docs/manuals/ibridges.qmd
chStaiger d4e24d1
Update docs/manuals/ibridges.qmd
chStaiger 000260a
Update docs/manuals/ibridges.qmd
chStaiger edebd17
restructuring and image
chStaiger 6813789
change order
jelletreep 8bbf0d7
update broken links
jelletreep File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -2,24 +2,107 @@ | |||||
title: Using iBridges on SURF Research Cloud | ||||||
--- | ||||||
|
||||||
The instructions below describe the process of data transfer between SURF Research Cloud and Yoda using iBridges. iBridges is a command-line tool for transfering data to and from Yoda (and other iRODS servers), focused on user-friendliness. It is developed and supported by the Utrecht University Research Engineering team. Like the [iCommands suite](icommands.qmd), it is suitable for transferring large amounts of data, but it is easier to use. | ||||||
The instructions below describe the how to setup iBridges to transfer data between SURF Research Cloud and Yoda or any other iRODS instance. It is possible to use iBridges in three different ways: | ||||||
|
||||||
Note: a graphical desktop version of iBridges is currently under development, but not yet supported on ResearchCloud. | ||||||
- as a python package | ||||||
- via the command line interface (CLI) | ||||||
- via the graphical user interface (GUI) | ||||||
|
||||||
|
||||||
|
||||||
Using any of the three methods you can safely and easily transfer large amounts of data between Yoda and your Research cloud workspace. | ||||||
|
||||||
It is developed and supported by the Utrecht University Research Engineering team. Like the [iCommands suite](icommands.qmd), it is suitable for transferring large amounts of data, but it is easier to use. | ||||||
|
||||||
The python package and the command line interface are available or can be made available on all workspaces while the graphical user interface will only work on 'Desktop' workspaces, i.e. workspaces where you will have a Windows or Linux (e.g. Ubuntu) Desktop. | ||||||
|
||||||
![ibridges](../imgs/ibridges.png) | ||||||
|
||||||
### Prerequisites | ||||||
|
||||||
You will need some basic Linux (commandline) skills to be able to use iBridges. | ||||||
Access to Yoda or another iRODS instance which hosts your data. | ||||||
|
||||||
iBridges needs to be installed on your workspace. Many SURF ResearchCloud workspaces developed by Utrecht University (such as Python and R workbenches) have iBridges preinstalled. | ||||||
|
||||||
- If you are working on a Desktop workspace, you can check by logging in to your workspace, clicking 'applications' in the top left corner, and clicking 'Development'. | ||||||
- On non-desktop workspaces check if iBridges is installed by typing: `ibridges` in the terminal. If iBridges is installed, this should show you the program's help menu. On Jupyter or Rstudio workspaces, you need to open a terminal first: In Jupyter, click the + button in the file browser and select the terminal in the new Launcher tab ([find a short video here](https://jupyterlab.readthedocs.io/en/stable/user/terminal.html)). | ||||||
In Rstudio, In the bottom left panel, click the ‘terminal’ tab. | ||||||
|
||||||
### Configure iBridges | ||||||
|
||||||
To connect to a Yoda or any other iRODS instance you will need a so-called `irods_environment.json` which needs to be stored in a specific location. | ||||||
|
||||||
#### Configuration through the CLI | ||||||
|
||||||
If you want to connect to a server hosted by Utrecht University please check whether this information is already present for ibridges. | ||||||
|
||||||
``` | ||||||
ibridges setup --list | ||||||
``` | ||||||
|
||||||
From that list simply choose the server you want to connect to with `ibridges setup uu-<server>` and follow the instructions. | ||||||
In case you get the message `No server information was found.` please install: | ||||||
|
||||||
``` | ||||||
pip install git+https://github.com/UtrechtUniversity/ibridges-servers-uu.git | ||||||
``` | ||||||
|
||||||
#### Configuration through the GUI | ||||||
|
||||||
1. Open the GUI | ||||||
- Windows: Go to the Main Menu and search for or click on the tile `ibridges` | ||||||
- Linux: Go to Applications --> Development --> iBridges | ||||||
|
||||||
2. From the main menu click Configure` --> `Add Configuration`. Click on the template you want to use, fill in your user name and save the file as `irods_environment.json` in the suggested `.irods` folder. | ||||||
|
||||||
iBridges is already installed on many SURF ResearchCloud workspaces developed by Utrecht University. You can check if it is present by simply typing the following command on your terminal: `ibridges`. If iBridges is installed, this should show you the program's help menu. | ||||||
If there are no templates in the drop-down menu of the configuration window, you will need to install the plugin. To this end you will need python and git and execute in power shell, git bash or Anaconda terminal: | ||||||
|
||||||
If you need to install iBridges, you can execute: `pip install ibridges`. | ||||||
``` | ||||||
pip install git+https://github.com/UtrechtUniversity/ibridges-servers-uu.git | ||||||
``` | ||||||
|
||||||
### Creating an iRODS environment file | ||||||
#### What shall I do if iBridges is not on my workspace? | ||||||
|
||||||
If you are confident with python and git you can install the packages: | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
- CLI/API: `pip install ibridges` | ||||||
- GUI: `pip install ibridgesgui` | ||||||
|
||||||
To start the GUI application you would then need to open a terminal and type in `ibridges-gui`. | ||||||
|
||||||
Please contact us if you need to install iBridges or if you have any trouble with the installation. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
|
||||||
#### How to connect to a Yoda or iRODS instance not provided by Yoda? | ||||||
|
||||||
In this case you will have to create your irods_environment.json by hand: | ||||||
|
||||||
{{< include partials/_irods_env.qmd >}} | ||||||
|
||||||
For a more extensive tutorial, see [here](https://github.com/UtrechtUniversity/iBridges/blob/develop/tutorials/01-Setup-and-connect.ipynb). | ||||||
For a more extensive tutorial, see | ||||||
|
||||||
- [Setup](https://github.com/UtrechtUniversity/iBridges/blob/develop/tutorials/01-Setup-and-connect.ipynb) | ||||||
|
||||||
- [Setup by GUI](https://chstaiger.github.io/iBridges-Gui/docs/getting-started.html) | ||||||
|
||||||
### Using iBridges | ||||||
|
||||||
For instructions on how to use the `ibridges` command please see the [iBridges documentation](https://github.com/UtrechtUniversity/iBridges?tab=readme-ov-file#commandline-interface). | ||||||
To use iBridges on Yoda hosted by Utrecht University you will need a [data access password](https://www.uu.nl/en/research/yoda/using-data-access-passwords). | ||||||
|
||||||
#### Command line interface | ||||||
|
||||||
Simply try `ibridges --help` or `ibridges <command> --help` to get examples and help for the usage. | ||||||
|
||||||
For instructions on how to use the `ibridges` command please see the [iBridges documentation](https://ibridges.readthedocs.io/en/stable/cli.html). | ||||||
|
||||||
#### Graphical user interface | ||||||
|
||||||
Once you have an `irods_environment.json`, click on `Connect` --> `Connect to iRODS`, choose your `irods_environment.json` and provide your data access password | ||||||
|
||||||
In the [iBridges GUI documentation](https://chstaiger.github.io/iBridges-Gui/docs/userdoc.html) | ||||||
|
||||||
#### Using the python package | ||||||
|
||||||
We offer [tutorials in form of Jupyter notebooks](https://github.com/UtrechtUniversity/iBridges/tree/main/tutorials) to show you the capabilities of the ibridges python package. | ||||||
|
||||||
For more information please see the [iBridges documentation](https://ibridges.readthedocs.io/en/stable/) |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.