This blog explains how to use Python SDK for publishing data products in SAP Business Data Cloud so that it becomes discoverable by other consumer applications in SAP Business Data Cloud, for example by SAP Datasphere.
Prerequisites – Adding Privileges into SAP Databricks
Steps to follow (Screenshots)
Access your SAP Databricks Workspace environment. From the sidebar menu, choose Catalog.Choose the Manage (Gear icon) button and, from the dropdown menu, select Metastore.Choose the Permissions tab and then choose the Grant button.In the Principals field, enter the user or a group of users.Select the boxes CREATE CATALOG, CREATE SHARE, SET SHARE PERMISSION, USE PROVIDER, USE RECIPIENT, and USE SHARE then choose Confirm.
Create a Delta Share
For a Delta Share to be exposed for downstream processing in SAP Databricks, it needs to be published as a data product with rich metadata. This can be done using an iPython Notebook available in your SAP Databricks Workspace. For further details on how to describe the data product, see Appendix A.
Steps to follow (Screenshots)
Access your SAP Databricks Workspace environment.From the sidebar menu, choose Catalog.Choose the Manage (Gear icon) button and, from the dropdown menu, select Delta sharing.Under Share data, provide the following information:Create share: In the Share name field, enter a share name and choose Save and continue. The Description field is optional.Add data assets: Select one or more schemas, which includes all the tables in the schema, or one or more individual tables. Choose Save and continue.Add notebooks: (Optional) Specify the Storage location. Choose Save and continue.Add recipients: In the Recipient field, from the dropdown menu, select sap-business-data-cloud. Choose Share data.The Delta Share is now exposed to the workspace.
Publishing the Delta Share as Data Product to SAP Business Data Cloud
Create a folder in the workspace by choosing Workspace Create Folder. Enter a name for this new folder and choose Create.Create a notebook, in the newly created folder, by selecting that folder and choosing Create Notebook.
Follow this steps from Learning Tutorial – https://learning.sap.com/courses/bt_bdc/dbx-enhance-data-product Update the serverless environment version by following these steps:On the sidebar menu, choose the Environment button.In the Environment version field, use the dropdown menu to choose 3.Choose Apply.Hover over the area under the rectangle and use +Code to create a cell in the notebook to run the instructions.In this cell, install sap-bdc-connect-sdk:
Installation
To install this SDK use the following pip command:
pip install sap-bdc-connect-sdk
uv tool install sap-bdc-connect-sdk
Python Documentation for BDC Connect – https://pypi.org/project/sap-bdc-connect-sdk/
Create or update the share in sap-bdc-connect-sdk.
A share is a mechanism for distributing and accessing data across different systems. Creating or updating a share involves including specific attributes, such as @openResourceDiscoveryV1, in the request body, aligning with the Open Resource Discovery protocol. This procedure ensures that the share is properly structured and described according to specified standards, facilitating effective data sharing and management.
Copy the below code snippet and paste it to the cell, then provide the share-name, title, shortDescription and description of your share.
from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, “<recipient-name>”))
share_name = “<share-name>”
open_resource_discovery_information = {
“@openResourceDiscoveryV1”: {
“title”: “<title>”,
“shortDescription”: “<short-description>”,
“description”: “<description>”
}
}
bdc_connect_client.create_or_update_share(
share_name,
open_resource_discovery_information
)
Create or update share CSN.
from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
from bdc_connect_sdk.utils import csn_generator
bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, “<recipient-name>”))
share_name = “<share-name>”
csn_schema = csn_generator.generate_csn_template(share_name)
bdc_connect_client.create_or_update_share_csn(
share_name,
csn_schema
)
If the generated CSN needs to be modified for any reason, it can be written to a file for editing, for example.
from bdc_connect_sdk.utils import csn_generator
import json
share_name = “<share-name>”
csn_schema = csn_generator.generate_csn_template(share_name)
file_path = f”csn_{share_name}.json”
with open(file_path, “w”) as csn_file:
csn_file.write(json.dumps(csn_schema, indent=2))
# change the file
with open(file_path, “r”) as csn_file:
changed_csn_schema = csn_file.read()
csn_schema = changed_csn_schema
Publish the data product.
from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, “<recipient-name>”))
share_name = “<share-name>”
bdc_connect_client.publish_data_product(
share_name
)
Unpublishing a Data Product
from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, “<recipient-name>”))
share_name = “<share-name>”
bdc_connect_client.delete_share(
share_name
)
This blog explains how to use Python SDK for publishing data products in SAP Business Data Cloud so that it becomes discoverable by other consumer applications in SAP Business Data Cloud, for example by SAP Datasphere.Prerequisites – Adding Privileges into SAP DatabricksSteps to follow (Screenshots)Access your SAP Databricks Workspace environment. From the sidebar menu, choose Catalog.Choose the Manage (Gear icon) button and, from the dropdown menu, select Metastore.Choose the Permissions tab and then choose the Grant button.In the Principals field, enter the user or a group of users.Select the boxes CREATE CATALOG, CREATE SHARE, SET SHARE PERMISSION, USE PROVIDER, USE RECIPIENT, and USE SHARE then choose Confirm.You have added the necessary privileges.Create a Delta ShareFor a Delta Share to be exposed for downstream processing in SAP Databricks, it needs to be published as a data product with rich metadata. This can be done using an iPython Notebook available in your SAP Databricks Workspace. For further details on how to describe the data product, see Appendix A.Steps to follow (Screenshots)Access your SAP Databricks Workspace environment.From the sidebar menu, choose Catalog.Choose the Manage (Gear icon) button and, from the dropdown menu, select Delta sharing.Under Share data, provide the following information:Create share: In the Share name field, enter a share name and choose Save and continue. The Description field is optional.Add data assets: Select one or more schemas, which includes all the tables in the schema, or one or more individual tables. Choose Save and continue.Add notebooks: (Optional) Specify the Storage location. Choose Save and continue.Add recipients: In the Recipient field, from the dropdown menu, select sap-business-data-cloud. Choose Share data.The Delta Share is now exposed to the workspace.Publishing the Delta Share as Data Product to SAP Business Data CloudCreate a folder in the workspace by choosing Workspace Create Folder. Enter a name for this new folder and choose Create.Create a notebook, in the newly created folder, by selecting that folder and choosing Create Notebook.Follow this steps from Learning Tutorial – https://learning.sap.com/courses/bt_bdc/dbx-enhance-data-product Update the serverless environment version by following these steps:On the sidebar menu, choose the Environment button.In the Environment version field, use the dropdown menu to choose 3.Choose Apply.Hover over the area under the rectangle and use +Code to create a cell in the notebook to run the instructions.In this cell, install sap-bdc-connect-sdk:InstallationTo install this SDK use the following pip command:pip install sap-bdc-connect-sdkuv tool install sap-bdc-connect-sdkPython Documentation for BDC Connect – https://pypi.org/project/sap-bdc-connect-sdk/ Create or update the share in sap-bdc-connect-sdk.A share is a mechanism for distributing and accessing data across different systems. Creating or updating a share involves including specific attributes, such as @openResourceDiscoveryV1, in the request body, aligning with the Open Resource Discovery protocol. This procedure ensures that the share is properly structured and described according to specified standards, facilitating effective data sharing and management.Copy the below code snippet and paste it to the cell, then provide the share-name, title, shortDescription and description of your share.from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, “<recipient-name>”))
share_name = “<share-name>”
open_resource_discovery_information = {
“@openResourceDiscoveryV1”: {
“title”: “<title>”,
“shortDescription”: “<short-description>”,
“description”: “<description>”
}
}
bdc_connect_client.create_or_update_share(
share_name,
open_resource_discovery_information
)Create or update share CSN.from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
from bdc_connect_sdk.utils import csn_generator
bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, “<recipient-name>”))
share_name = “<share-name>”
csn_schema = csn_generator.generate_csn_template(share_name)
bdc_connect_client.create_or_update_share_csn(
share_name,
csn_schema
)If the generated CSN needs to be modified for any reason, it can be written to a file for editing, for example.
from bdc_connect_sdk.utils import csn_generator
import json
share_name = “<share-name>”
csn_schema = csn_generator.generate_csn_template(share_name)
file_path = f”csn_{share_name}.json”
with open(file_path, “w”) as csn_file:
csn_file.write(json.dumps(csn_schema, indent=2))
# change the file
with open(file_path, “r”) as csn_file:
changed_csn_schema = csn_file.read()
csn_schema = changed_csn_schema
Publish the data product.from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, “<recipient-name>”))
share_name = “<share-name>”
bdc_connect_client.publish_data_product(
share_name
)Unpublishing a Data Productfrom bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, “<recipient-name>”))
share_name = “<share-name>”
bdc_connect_client.delete_share(
share_name
) Read More Technology Blog Posts by SAP articles
#SAP
#SAPTechnologyblog
