BDM - Connection

> Data Integration Tool (ETL/ELT) > Big Data Management Informatica (BDM)

1 - About

BDM connections:

  • HBase (a NoSQL database comprising key-value pairs on Hadoop that performs operations in real-time)
  • Hadoop,
  • HDFS,
  • Hive connections
  • JDBC

3 - Managment

With BDM - infacmd

infacmd isp --help
Connection commands
  addConnectionPermissions: Assigns connection permissions to a user or a group.
  createConnection: Defines a connection and the connection options.
  generateHadoopConnectionFromHiveConnection: Generates Hadoop connection from Hive pushdown connections.
  listConnectionOptions: Lists options for a connection.
  listConnectionPermissions: Lists the permissions that a user or group has for a connection.
  listConnectionPermissionsByGroup: Lists all groups that have permissions on a connection and lists the type of permissions.
  listConnectionPermissionsByUser: Lists all users that have permissions on a connection and lists the type of permissions.
  listConnections: Lists connection names by type.
  removeConnection: Removes a connection from the domain.
  removeConnectionPermissions: Removes connection permissions for a user or a group.
  renameConnection: Renames a connection.
  setConnectionPermissions: Assigns permissions on connection to a user or a group after removing previous permissions.
  updateConnection: Updates a connection.
Advertising

3.1 - Options

List the properties / parameters/ options of a connection

infacmd.sh isp listConnectionOptions \
  -cn connectionName \
  -DomainName $INFA_DOMAIN_NAME \
  -UserName $INFA_USERNAME \
  -Password $INFA_PASSWORD \
  -sdn $INFA_SECURITY_DOMAIN 

Example with a Hive Connection:

connectionId: [HIVE_HDI_BDM_DEV]
connectionType: [HIVE]
name: [HIVE_HDI]
connectString: [jdbc:hive2://hn0-blabla:10001/default;transportMode=http;httpPath=/;saslQop=auth;principal=hive/[email protected]]
enableQuotes: [false]
environmentSQL: []
metadataConnString: [jdbc:hive2://hn0-blabla:10001/default;transportMode=http;httpPath=/;saslQop=auth;principal=hive/[email protected]]
password: [**********]
quoteChar: [BACK_QUOTE]
userName: [[email protected]]
bypassHiveJDBCServer: [false]
clusterConfigId: [hdi_bdm_dev]
hadoopClusterName: []
hiveJdbcDriverClassName: [org.apache.hive.jdbc.HiveDriver]
sqlAuthorized: [false]

3.2 - Query

with BDM - Model Repository Service (MRS)

  • Connection used in mapping
SELECT CONN_NAME, COUNT(1) FROM MRX_INSTANCE_SOURCES WHERE CONN_NAME IS NOT NULL GROUP BY conn_name ORDER BY CONN_NAME;
  • Connection defined in Domain
SELECT DISTINCT PSC_DISCRIMINATOR, PSC_EXTERNID FROM dbo.PO_CONNECTINFO ORDER BY PSC_EXTERNID;