VRE #4894

BlueBridge RStudio VRE

Added by Julien Barde over 2 years ago. Updated over 1 year ago.

Status:ReleasedStart date:Aug 22, 2016
Priority:UrgentDue date:Mar 28, 2017
Assignee:Massimiliano Assante% Done:

100%

Sprint:WP08
VRE_Policy:Open Invite_Policy:Open
Milestones:
Duration: 157

Description

By discussing with some colleagues, we think that having a very basic VRE only made of RStudio (and others default services: workspace and messages) might be a good way to showcase the benefits of working with an infrastructure like BlueBridge and a first step to set up more complicated VREs. RStudio online is a very useful service, already in the infra which is used now in multiple VREs but it's tricky to access it and it deserves to be accessible in a straightforward way (where people who want to use it need to go through specific VREs for now). We think RStudio might be a good corner stone to set up more specific VREs in a second step. Currently RStudio is still hidden and it's not simple to use it although multiple users would probably be keen to try it.
I think it might be a good way as well to advertise the fact that BlueBridge as an infra can be used in multiple training sessions.

  • VRE name and abstract: BlueBridge RStudio. If possible I will later advertise this VRE (my research unit, tuna commissions..)as a way to drive collaborative work / training courses without facing usual R configurations issues (R versions and packages),
  • Membership policy: open to anybody in the domain if possible.
  • VRE expected datasets: no requirements at the beginning but we can guess that users might be interested to store / make datasets available in the infra once used to work with BlueBridge RStudio,
  • VRE expected functionalities which would be useful for all VREs in the medium term:
    • a dedicated URL enables users to login and access directly to this VRE (same for any VRE),
    • users can open some worspace documents /repositories from RStudio

Subtasks

D4Science Infrastructure - Task #4865: Installing data analysis machine to develop prototypesClosedAndrea Dell'Amico

Task #5465: ShareLatexClosedAndrea Dell'Amico

Task #7089: Connect the sharelatex users to the ldap onesClosedLucio Lelii

Project Activity #7437: Connect Rserver to sharelatex to compile knitr codesClosedTAHA IMZILEN

Task #7612: sharelatex needs to be linked to some VREClosedMassimiliano Assante

Task #7862: Enable ShareLaTex on gCubeApps/RStudioLabClosedRoberto Cirillo

Support #8799: Enable sharelatex on ICCAT BFT-E and Tuna AtlasClosedJulien Barde

Task #5466: enable the shiny package within RStudio serverClosedAndrea Dell'Amico


Related issues

Related to BlueBRIDGE - Project Task #631: T4.2 VREs Creation and Deployment [Months: 1-30] Closed Feb 02, 2016 Nov 10, 2017

History

#1 Updated by Pasquale Pagano over 2 years ago

  • Invite_Policy set to Open
  • VRE_Policy set to Open
  • Due date set to Sep 09, 2016
  • Tracker changed from Support to VRE

#2 Updated by Leonardo Candela over 2 years ago

I would suggest to reconsider the name of this VRE, what about BlueBRIDGE R IDE or simply R IDE? This is because RStudio refers only to a piece (although the major one) of the VRE facilities.

My second comment is about the lack of a description, what about the following one:

This Virtual Research Environment is conceived to provide its users with a development environment for R. This development environment includes an RStudio instance and it is integrated with social networking facilities and a shared work space enabling VRE members to put in place collaborative working practices.

Not clear to me whether we are going to offer StatMan/Dataminer (and SAI) also.

PS: Italian world "ride" means smile but this is not important.

#3 Updated by Julien Barde over 2 years ago

In this case I was deliberately suggesting a VRE made only of RStudio and nothing else which would be great for newcomers (and others) to clarify the benefits of VREs. As far as I know such a service doesn't exist with packages for marine domaine set up and "open" access.

Once they become users of this simple VRE, I think they will be able to realize potential benefits of more specific VREs with additionnal services. This is why I wanted RStudio in the name so that they will remind it. Moreover, all users will know what RStudio means but I am afraid that only few of them will know what IDE means. If RStudio is not the only R tool then it would be worth to refer to R rather than RStudio in the title. For example if RShiny could be enabled later or tools like Sharelatex which enables compilation of knitr documents. R Toolbox might be another option. I wanted BlueBridge or any other word to indicate this is related to marine domaine.

To summarize, I would suggest to try it first with RStudio only (including workspace and social network), see how it works and then discuss the integration of additionnal services thereafter depending on the success and feedback of first users. I think it's easier to add items in the menu once users are already used to the tool. If there is too much information first then they struggle to find the VRE undestand the different items..I experienced that with multiple colleagues.

If we can have a URL dedicated to Rstudio we think it would help a lot to adopt it as a regular tool.

The description is fine as this is indeed about using a "very simple" VRE / collaborative environment. We could specify that RStudio comes with system packages widely used by users in the marine domain. If possible that the list of packages could be udpated according to the feedback of users. We can explain as well that this is a good way to dirve courses and ensure that collaborative codes will find a place where the compilation will be successful and with descent machine resources...

#4 Updated by Emmanuel Blondel over 2 years ago

Agreed on the fact 'IDE' is not meaningful for many users. +1 for mentioning RStudio, like BlueBridge or 'Blue RStudio'. I also think that other services should be dropped there later depending on actual needs.

#5 Updated by Pasquale Pagano over 2 years ago

What about RStudioLab?

#6 Updated by Julien Barde over 2 years ago

This name is OK for me.

Just to let you know, I just made a try few minutes ago to connect a colleague of Ifremer to the stock assessment VRE because he just wants to use RStudio. He's bot really interested in other services at this stage.

I confirm that it's confusing for a newcomer to understand how to become a user and what VRE should be chosen. By the way I can't see the pending request in the VRE.

Another issue for the link from the official BlueBridge Website. If you google "stock assessment VRE", the first result is
http://www.bluebridge-vres.eu/services/stock-assessment
If you click on the link you arrive there
https://i-marine.d4science.org/web/guest/virtual-research-environments which is a wrong URL (instead of https://i-marine.d4science.org/group/stockassessment)

#7 Updated by Leonardo Candela over 2 years ago

I'm not an expert in legal issues that's why I suggested to drop RStudio from the name.

After reading here https://www.rstudio.com/about/trademark/ my feeling is that our scenario falls in the "Automatic Permitted Use" of the trademark.

However, this has some implications ... if we go for maintaining RStudio in the name I suggest to add something in the name to comply with the RStudio expectations, e.g. add this sentence in the VRE description "RStudio [and Shiny] are trademarks of RStudio, Inc.".

Re the rest of issues you raised, e.g. the wrong link I'll inform the Trust-IT colleagues.

#8 Updated by Julien Barde over 2 years ago

Another comment: I often try to re-connect RStudio by directly using this URL http://rstudio.d4science.org/

When disconnected I obtain http://rstudio.d4science.org/r-connector/gcube/service/disconnect

and the following message:
Rsutdio session inactive log in using TabularData portlet from the i-marine portal

Could we reconnect Rstudio directly by having the usual login interface instead of this message ?

#9 Updated by Pasquale Pagano over 2 years ago

Julien Barde wrote:

Another comment: I often try to re-connect RStudio by directly using this URL http://rstudio.d4science.org/

When disconnected I obtain http://rstudio.d4science.org/r-connector/gcube/service/disconnect

and the following message:
Rsutdio session inactive log in using TabularData portlet from the i-marine portal
We can improve this message.

Could we reconnect Rstudio directly by having the usual login interface instead of this message ?

Not really since the user is registered in the VRE and not in RStudio. The user of the VRE is mapped on the RStudio user but the user cannot know how to connect to it. When you connect to the VRE you get a 'token' that is used to connect to RStudio. So, you cannot use the usual login interface. @lucio.lelii@isti.cnr.it can you confirm please?

#10 Updated by Lucio Lelii over 2 years ago

Yes, it work in this way.
We will improve the disconnect message.

#11 Updated by Emmanuel Blondel over 2 years ago

Isn't there a way, in case the user goes to http:/rstudio.d4science.org to redirect to VRE login, and then once logged in to go back to RStudio?

#12 Updated by Pasquale Pagano over 2 years ago

It is a good suggestion. This virtual RStudio server is shared between several VREs. We will improve the message linking to the portal site for the login.

#13 Updated by Leonardo Candela over 2 years ago

  • Status changed from New to Planned

#14 Updated by Julien Barde over 2 years ago

As the same RStudio server is used by multiple VREs, I am using this ticket to share some of the discussions we had here to improve the use or Rstudio.

We think that something BlueBridge can promote is the fact that users from multiple institutes are going to use this RStudio server. De facto, by collecting the feedback of users, step by step, the BlueBridge RStudio server will gather requirements to configure properly R in the marine domain. For example, I have been sharing some R codes with a colleague yesterday and I realized that "ncdf4" package is not a system package on BlueBridge RStudio, I installed it few months ago as a user package but didn't remember it and my code didn't work immediatly on another workspace. Such a package should be a default package for any user. I have other examples of packages which could be good candidates for default system packages (gdalUtils, shapefiles, ggmap, marmap...). At the end of the project, it would be a valuable legacy to share the specifications of a typical RStudio for marine domain (as it will reflect the needs of users in the marine domain).

We confirm that, if possible, it would be great as well to link the workspace files or repositories with RStudio (rights to read at least). If this is the relevant ticket and if interesting for the project of course, we will use it to provide this kind of feedback.

#15 Updated by Leonardo Candela over 2 years ago

  • Status changed from Planned to In Progress

#16 Updated by Gianpaolo Coro over 2 years ago

Dear Julien,
we are already collecting all the packages required by the BlueBRIDGE applications and we have created automatic installation scripts to create new RStudio environment.
The complete list of packages is available here #2174. Thus, we have an overview now of the ecosystem of packages required for the running applications. When you indicate one package in the SAI interface, we add this package to the overall list.

On the other hand, we have prepared a powerful prototyping machine to support RStudio development, which also hosts a Dataminer to directly test the algorithms (#4865).

We now need to create the RStudio VRE, where this RStudio instance will be added and tested. In a first version, Dataminer and SAI can be there too. This can be the development environment for your algorithms, which would avoid that you use the development environment. Could this be suited to your request for the time being? We could later disconnect Dataminer or make it visible only to users with higher privileges.

@leonardo.candela@isti.cnr.it is it possible to create the RStudio VRE?

#17 Updated by Julien Barde over 2 years ago

I am personally using Stock Assessment VRE SAI to deploy algorithms. If RStudio is the same for all VREs then we will have access to it from any VRE and we know that RStudio and Dataminer share the same configuration now.

So, I don't think we need Dataminer in this VRE. Moreover, at this stage, deploying algorithms is too complicated for normal users. It's not trivial and I think it's better to keep the VRE simple first and then go further step by step. For me the reason why would people user SAI is to enable the execution of an algorithm through Web forms or Web Services. We will need to demonstrate the interest of the approach and having the SAI in the VRE is not necessary. However if you can make it visible for some users this if OK of course. This way it's going to be hidden and once people understand why they want to access it they can ask.

However, my feeling is that people will ask us to do it first (they will need some help and guidelines any way) and perhaps very few of them will try. Remember that we (Taha, Paul and myself) are not the representing the target / community of users.

If you can enable to run local (only from users workspaces) shiny applications from RStudio it would be very nice.

Could you as well describe what are the machines resources available? Users will be interested in knowing it.

I discussed this morning with a colleague who said some colleagues of JRC might be interested to try it depending on packages installed and machines resources. Some are asking access to Ifremer supercomputer but would prefer RStudio VRE I guess if they have decent machine resources.

#18 Updated by Leonardo Candela over 2 years ago

  • Status changed from In Progress to Available

The RStudioLab is finally available https://i-marine.d4science.org/group/rstudiolab

@julien.barde@ird.fr is the manager of this environment. Please, let us know if you are willing to modify something.

#19 Updated by Leonardo Candela over 2 years ago

  • Status changed from Available to Released

#20 Updated by Julien Barde over 2 years ago

Did you check if why it's not possible to use Rshiny within RStudio (it's possible within RStudio Server)
For example by compiling this code in RStudio:
library(shiny)
runExample("01_hello")

A RShuny application should be displayed. This is almost OK but doesn't work properly

Another question regarding Sharelatex: this would be very usefull for collaborative edition of Latex / knitr documents.
Would it be possible to set up Sharelatex by using the same RServer as the one used by other applications ? https://github.com/sharelatex/sharelatex
This would enable collaborative edition of latex source code and compilation of the document with proper packages and decent machine resoruces as the freely accessible online Sharelatex server doesn't enable to compile the kind of documents we need (time execution and R packages are restricted).

#21 Updated by Massimiliano Assante over 2 years ago

  • Related to Project Task #631: T4.2 VREs Creation and Deployment [Months: 1-30] added

#22 Updated by Pasquale Pagano over 2 years ago

Julien Barde wrote:

Did you check if why it's not possible to use Rshiny within RStudio (it's possible within RStudio Server)
For example by compiling this code in RStudio:
library(shiny)
runExample("01_hello")

A RShuny application should be displayed. This is almost OK but doesn't work properly

RShiny is not open source. It costs close to 10 thousand euro per year for 20 concurrent users and the cost increase if you need to manage more users (+5 thousand for additional 20 users and +15 thousand for additional 150 users). The infrastructure cannot afford these license costs. Clearly, if the community want to buy a license for themselves we are ready to host and operate it for the exclusive operation of the community paying for it.

Another question regarding Sharelatex: this would be very usefull for collaborative edition of Latex / knitr documents.
Would it be possible to set up Sharelatex by using the same RServer as the one used by other applications ? https://github.com/sharelatex/sharelatex
This would enable collaborative edition of latex source code and compilation of the document with proper packages and decent machine resoruces as the freely accessible online Sharelatex server doesn't enable to compile the kind of documents we need (time execution and R packages are restricted).

I don't know ShareLatex but looking at the web I see that ShareLatex.com offers the service with the following prices: a personal license is free but a Collaborator license (up to 10 collaborators per project) costs 168 euro per year and a Professional license (unlimited collaborators) costs 336 euro per year.
Are you asking to deliver a similar service as sharelatex.com but for free in the VRE context?

#23 Updated by Emmanuel Blondel over 2 years ago

To feed the discussion here: i received 2 interesting feedback from a FAO R user, for BlueBridge RStudio VRE (all RStudio instances in general, in particular the RPrototypingLab that we use at now):

  • synchronize virtual workspace and Rstudio workspace, as highlighted at https://support.d4science.org/issues/4918#note-5 This mix of workspace is first source of confusion for users, and a limitation for R users.

  • enable the RStudio version-control module (GIT/Subversion)
    is it possible to have enabled the RStudio Version control (Git, Subversion)? So people that maintain their codes in a Git repository could have their working code there and the capacity to synchronize with their repo? It's really useful to have this kind of synchronization UI in RStudio and this vs. SAI, since the former is the place where people execute code and actually work and test their business R scripts, while the latter is only to register and publish an algorithm, not for simulation.
    It seems the version-control component of RStudio is not limited to Rstudio desktop, but can be enabled in RStudio server (if version-control GIT/Subversions are installed on the machine). Thanks in advance if you can enable it.

#24 Updated by Julien Barde over 2 years ago

@pasquale.pagano@isti.cnr.it
I am not asking to set up RShiny Server. What I am asking for is to enable the shiny package within RStudio server and this is different.
Just like plotly which is a product of a company, RShiny is entirely free if you use it in your RStudio Workspace because shiny or plotly products stay in your workspace. Rshiny server is required to host Shiny applications and make them public for many users with URLs out of your workspace.

So I am not sure if RShiny server is free or not (even if it sounds it can be free here https://www.rstudio.com/products/shiny/download-server/) but I think it doesn't matter for what we are asking for here, I am almost sure the Rshiny package within RStudio is free (as we have a RStudio Server in our Intranet and it works well with RShiny package). If you try to compile the code I sent, you will see that it almost work in the infra. Perhaps something is missing because by default the output can be browsed with the IP of RStudio server (to build the URL) and in your case you are using another domain name.

Regarding ShareLatex, it's the same as plotly, Rshiny and similar services. You will pay if you rely on their machine resources to host or process your data and applications but if you install on your server it then, if my understanding is correct of course, it should be free just like RStudio Server with limited services (compared to pro version). It's even more interesting in our case because RServer is already there behind the scene with expected packages on the infra.
As far as I understand www.sharelatex.com is just an instance of sharelatex project (https://github.com/sharelatex/sharelatex) and it sounds you can set up your sharelatex server for free: https://www.sharelatex.com/university/onsite.html only the "pro" version has a cost. However I might be wrong.

#25 Updated by Pasquale Pagano over 2 years ago

Julien Barde wrote:

@pasquale.pagano@isti.cnr.it
I am not asking to set up RShiny Server. What I am asking for is to enable the shiny package within RStudio server and this is different.
Just like plotly which is a product of a company, RShiny is entirely free if you use it in your RStudio Workspace because shiny or plotly products stay in your workspace. Rshiny server is required to host Shiny applications and make them public for many users with URLs out of your workspace.

I am going to open a ticket for shiny package. Making it available (if it works) will enable sharing across VRE users and this is something like an instance of Shiny server. However, let's proceed step by step. In the new sub ticket we will verify the technical feasibility and in the meanwhile I will try to understand a bit better the license to avoid that we violate some policies.

So I am not sure if RShiny server is free or not (even if it sounds it can be free here https://www.rstudio.com/products/shiny/download-server/)

It is not. The free version has severe limitations in terms of exploitation that make impossible to use in our context.

but I think it doesn't matter for what we are asking for here, I am almost sure the Rshiny package within RStudio is free (as we have a RStudio Server in our Intranet and it works well with RShiny package). If you try to compile the code I sent, you will see that it almost work in the infra. Perhaps something is missing because by default the output can be browsed with the IP of RStudio server (to build the URL) and in your case you are using another domain name.

We will do it.

Regarding ShareLatex, it's the same as plotly, Rshiny and similar services. You will pay if you rely on their machine resources to host or process your data and applications but if you install on your server it then, if my understanding is correct of course, it should be free just like RStudio Server with limited services (compared to pro version). It's even more interesting in our case because RServer is already there behind the scene with expected packages on the infra.
As far as I understand www.sharelatex.com is just an instance of sharelatex project (https://github.com/sharelatex/sharelatex) and it sounds you can set up your sharelatex server for free: https://www.sharelatex.com/university/onsite.html only the "pro" version has a cost. However I might be wrong.

I opened a ticket also for this case.

Stay tuned and let's see if we can satisfy both requests.

#26 Updated by Andrea Dell'Amico over 2 years ago

Emmanuel Blondel wrote:

  • enable the RStudio version-control module (GIT/Subversion) is it possible to have enabled the RStudio Version control (Git, Subversion)? So people that maintain their codes in a Git repository could have their working code there and the capacity to synchronize with their repo? It's really useful to have this kind of synchronization UI in RStudio and this vs. SAI, since the former is the place where people execute code and actually work and test their business R scripts, while the latter is only to register and publish an algorithm, not for simulation. It seems the version-control component of RStudio is not limited to Rstudio desktop, but can be enabled in RStudio server (if version-control GIT/Subversions are installed on the machine). Thanks in advance if you can enable it.

The rstudio version control support is already active. On rstudio/rstudio-dev the git and subversion clients where not installed but they are now present.
You can find the options under Tools -> Global Options.

In the near future we will also activate a local git server instance and make it available on each VRE that needs it.

#27 Updated by Andrea Dell'Amico over 2 years ago

  • Due date set to Feb 13, 2017

due to changes in a related task

Also available in: Atom PDF