Custom Roles Management

Learn how to create and manage custom roles in Diffgram.

Diffgram has a built in Role Manager so that you can define, assign and manage roles with custom permissions to different objects in your project. The main objects that support custom roles management are the Project and the Dataset objects, but future releases will support new objects for permissions management.

Check the Notebook version of this guide here: https://colab.research.google.com/drive/1k_nHEIzjGK1_6JOsMGcZ9J2MAPKoBsN_#scrollTo=TFyZAv6gup_s

Getting Started

In this guide we will show you how to create, update and delete custom roles and permissions using the Python SDK. All of the SDK calls can be done in any other language by using our direct HTTP API calls.

Example Situation

• Let's say we have a Project called Sleetduke. On this project we want to have a couple of "hidden" datasets. This dataset can only be accessed if we explictly grant access to it. In this case the permissions will be view only.

• On the other hand, we would like to have a role called "security_advisor". The security advisor role will be able to see all the hidden datasets, and also edit them.

Let's see how we can model this situation using the Diffgram SDK and custom roles.

1. Connect your project

Let's connect to our project. Make sure the client secret and ID have the admin role on the project. This will allow you to be able to create datasets.

Python

from diffgram.core.core import Project
project = Project(
            project_string_id = "replace_with_your_project_id",
            client_id = "replace_with_your_client_id",
            client_secret = "replace_with_your_client_secret",
        )

2. Create some datasets

Now we can create 5 datasets. 3 of them will be accessible by everyone in the project, and 2 of them will have "restricted" access. Dataset that are restricted can only be viewed if a user if a user has a role that grants the "dataset_view" permission over the given dataset.

Python

# Create the project-accessible datasets.
ds1 = project.directory.new(name='Dataset 1', access_type='project')
ds2 = project.directory.new(name='Dataset 2', access_type='project')
ds3 = project.directory.new(name='Dataset 3', access_type='project')
# Create datasets with restricted access
restricted_ds1 = project.directory.new(name='Hidden Dataset 1', access_type='restricted')
restricted_ds2 = project.directory.new(name='Hidden Dataset 2', access_type='restricted')

Let's go to the Diffgram Studio on the UI and refresh the dataset selector. If you have the admin role you should be able to see all datasets.

But if you access the dataset with the editor role, you will only see the datasets that are not hidden.

3. Create a Custom Role
   Now let's create a custom role. This role should be able to access the hidden datasets.

Python

advisor_role = project.roles.new(name='security_advisor')

If you get a permissions error. Make sure that you Client Secret and ID was generated with the admin role. This is the only role that can manage permissions and create custom roles in a project.

4. Assign Permissions to role

Now we can assign permissions to the created role. In this case we want to make the role have the dataset_view permission. This will allow it to see datasets that we assign the role to. You can see the other permissions available on the Default Roles Section

Python

advisor_role.add_permission(perm='dataset_view', object_type='WorkingDir')

5. Assign Roles to users and datasets.

Now that our role has a new permission, we can assign it to a user. In this case we want to assign this role to the hidden datasets and the user that we grant acesss to. You can imagine the assignment as a three value tupple [object_id, role, diffgram_member_id]. Replace the email with an email from your team that is already in the Diffgram project.

Python

member_to_grant = project.get_member(email='[email protected]')
advisor_role.assign_to_member_in_object(member_id=member.get('member_id'), object_id=restricted_ds1.id, object_type='WorkingDir')
advisor_role.assign_to_member_in_object(member_id=member.get('member_id'), object_id=restricted_ds2.id, object_type='WorkingDir')

Now you should be able to see the datasets! As easy as that you can control access to your data on your ML pipelines.

Now as a last step, lets remove the role assignments.

6. Remove Role Assignments

Python

advisor_role.remove_role_assignment(member_id=member.get('member_id'), object_id=restricted_ds1.id, object_type='WorkingDir')
advisor_role.remove_role_assignment(member_id=member.get('member_id'), object_id=restricted_ds2.id, object_type='WorkingDir')

That's it!

This is a small demonstration of what you can do to control permissions and access to datasets in Diffgram. Feel free to open an issue on github or shoot a message on our slack if you have any questions.