Returns the partitions Resource.
Close httplib2 connections.
create(parent, body=None, validateOnly=None, x__xgafv=None)
Create a metadata entity.
delete(name, etag=None, x__xgafv=None)
Delete a metadata entity.
get(name, view=None, x__xgafv=None)
Get a metadata entity.
list(parent, filter=None, pageSize=None, pageToken=None, view=None, x__xgafv=None)
List metadata entities in a zone.
list_next(previous_request, previous_response)
Retrieves the next page of results.
update(name, body=None, validateOnly=None, x__xgafv=None)
Update a metadata entity. Only supports full resource update.
close()
Close httplib2 connections.
create(parent, body=None, validateOnly=None, x__xgafv=None)
Create a metadata entity. Args: parent: string, Required. The resource name of the parent zone: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}. (required) body: object, The request body. The object takes the form of: { # Represents tables and fileset metadata contained within a zone. "asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset. "catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry. "compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with. "bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, "hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, }, "createTime": "A String", # Output only. The time when the entity was created. "dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id. "dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv. "description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters. "displayName": "A String", # Optional. Display name must be shorter than or equal to 63 characters. "etag": "A String", # Optional. The etag for this entity. Required for update and delete requests. Must match the server's etag. "format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery. "compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed. "csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data. "delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified. "headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0. "quote": "A String", # Optional. The character used to quote column values. Accepts '"' and '''. Defaults to '"' if unspecified. }, "format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type. "json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified. }, "mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: - application/x-parquet - application/x-avro - application/x-orc - application/x-tfrecord - application/json - application/{subtypes} - text/csv - text/ - image/{image subtype} - video/{video subtype} - audio/{audio subtype} }, "id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores. Must begin with a letter. "name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}. "schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response. "fields": [ # Optional. The sequence of fields describing data in table entities. { # Represents a column field within a table schema. "description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters. "fields": [ # Optional. Any nested field for complex types. # Object with schema name: GoogleCloudDataplexV1SchemaSchemaField ], "mode": "A String", # Required. Additional field semantics. "name": "A String", # Required. The name of the field. The maximum length is 767 characters. The name must begins with a letter and not contains : and .. "type": "A String", # Required. The type of field. }, ], "partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data. { # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. "name": "A String", # Required. Partition name is editable if only the partition style is not HIVE compatible. The maximum length allowed is 767 characters. "type": "A String", # Required. Immutable. The type of field. }, ], "partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity. "userManaged": True or False, # Required. Whether the schema is user-managed or managed by the service. - Set user_manage to false if you would like Dataplex to help you manage the schema. You will get the full service provided by Dataplex discovery, including new data discovery, schema inference and schema evolution. You can still provide input the schema of the entities, for example renaming a schema field, changing CSV or Json options if you think the discovered values are not as accurate. Dataplex will consider your input as the initial schema (as if they were produced by the previous discovery run), and will evolve schema or flag actions based on that. - Set user_manage to true if you would like to fully manage the entity schema by yourself. This is useful when you would like to manually specify the schema for a table. In this case, the schema defined by the user is guaranteed to be kept unchanged and would not be overwritten. But this also means Dataplex will not provide schema evolution management for you. Dataplex will still be able to manage partition registration (i.e., keeping the list of partitions up to date) when Dataplex discovery is turned on and user_managed is set to true. }, "system": "A String", # Required. Immutable. Identifies the storage system of the entity data. "type": "A String", # Required. Immutable. The type of entity. "updateTime": "A String", # Output only. The time when the entity was last updated. } validateOnly: boolean, Optional. Only validate the request, but do not perform mutations. The default is false. x__xgafv: string, V1 error format. Allowed values 1 - v1 error format 2 - v2 error format Returns: An object of the form: { # Represents tables and fileset metadata contained within a zone. "asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset. "catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry. "compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with. "bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, "hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, }, "createTime": "A String", # Output only. The time when the entity was created. "dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id. "dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv. "description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters. "displayName": "A String", # Optional. Display name must be shorter than or equal to 63 characters. "etag": "A String", # Optional. The etag for this entity. Required for update and delete requests. Must match the server's etag. "format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery. "compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed. "csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data. "delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified. "headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0. "quote": "A String", # Optional. The character used to quote column values. Accepts '"' and '''. Defaults to '"' if unspecified. }, "format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type. "json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified. }, "mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: - application/x-parquet - application/x-avro - application/x-orc - application/x-tfrecord - application/json - application/{subtypes} - text/csv - text/ - image/{image subtype} - video/{video subtype} - audio/{audio subtype} }, "id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores. Must begin with a letter. "name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}. "schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response. "fields": [ # Optional. The sequence of fields describing data in table entities. { # Represents a column field within a table schema. "description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters. "fields": [ # Optional. Any nested field for complex types. # Object with schema name: GoogleCloudDataplexV1SchemaSchemaField ], "mode": "A String", # Required. Additional field semantics. "name": "A String", # Required. The name of the field. The maximum length is 767 characters. The name must begins with a letter and not contains : and .. "type": "A String", # Required. The type of field. }, ], "partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data. { # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. "name": "A String", # Required. Partition name is editable if only the partition style is not HIVE compatible. The maximum length allowed is 767 characters. "type": "A String", # Required. Immutable. The type of field. }, ], "partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity. "userManaged": True or False, # Required. Whether the schema is user-managed or managed by the service. - Set user_manage to false if you would like Dataplex to help you manage the schema. You will get the full service provided by Dataplex discovery, including new data discovery, schema inference and schema evolution. You can still provide input the schema of the entities, for example renaming a schema field, changing CSV or Json options if you think the discovered values are not as accurate. Dataplex will consider your input as the initial schema (as if they were produced by the previous discovery run), and will evolve schema or flag actions based on that. - Set user_manage to true if you would like to fully manage the entity schema by yourself. This is useful when you would like to manually specify the schema for a table. In this case, the schema defined by the user is guaranteed to be kept unchanged and would not be overwritten. But this also means Dataplex will not provide schema evolution management for you. Dataplex will still be able to manage partition registration (i.e., keeping the list of partitions up to date) when Dataplex discovery is turned on and user_managed is set to true. }, "system": "A String", # Required. Immutable. Identifies the storage system of the entity data. "type": "A String", # Required. Immutable. The type of entity. "updateTime": "A String", # Output only. The time when the entity was last updated. }
delete(name, etag=None, x__xgafv=None)
Delete a metadata entity. Args: name: string, Required. The resource name of the entity: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{entity_id}. (required) etag: string, Required. The etag associated with the partition if it was previously retrieved. x__xgafv: string, V1 error format. Allowed values 1 - v1 error format 2 - v2 error format Returns: An object of the form: { # A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } The JSON representation for Empty is empty JSON object {}. }
get(name, view=None, x__xgafv=None)
Get a metadata entity. Args: name: string, Required. The resource name of the entity: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{entity_id}. (required) view: string, Optional. Used to select the subset of entity information to return. Defaults to BASIC. Allowed values ENTITY_VIEW_UNSPECIFIED - The API will default to the BASIC view. BASIC - Minimal view that does not include the schema. SCHEMA - Include basic information and schema. FULL - Include everything. Currently, this is the same as the SCHEMA view. x__xgafv: string, V1 error format. Allowed values 1 - v1 error format 2 - v2 error format Returns: An object of the form: { # Represents tables and fileset metadata contained within a zone. "asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset. "catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry. "compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with. "bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, "hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, }, "createTime": "A String", # Output only. The time when the entity was created. "dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id. "dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv. "description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters. "displayName": "A String", # Optional. Display name must be shorter than or equal to 63 characters. "etag": "A String", # Optional. The etag for this entity. Required for update and delete requests. Must match the server's etag. "format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery. "compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed. "csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data. "delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified. "headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0. "quote": "A String", # Optional. The character used to quote column values. Accepts '"' and '''. Defaults to '"' if unspecified. }, "format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type. "json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified. }, "mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: - application/x-parquet - application/x-avro - application/x-orc - application/x-tfrecord - application/json - application/{subtypes} - text/csv - text/ - image/{image subtype} - video/{video subtype} - audio/{audio subtype} }, "id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores. Must begin with a letter. "name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}. "schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response. "fields": [ # Optional. The sequence of fields describing data in table entities. { # Represents a column field within a table schema. "description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters. "fields": [ # Optional. Any nested field for complex types. # Object with schema name: GoogleCloudDataplexV1SchemaSchemaField ], "mode": "A String", # Required. Additional field semantics. "name": "A String", # Required. The name of the field. The maximum length is 767 characters. The name must begins with a letter and not contains : and .. "type": "A String", # Required. The type of field. }, ], "partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data. { # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. "name": "A String", # Required. Partition name is editable if only the partition style is not HIVE compatible. The maximum length allowed is 767 characters. "type": "A String", # Required. Immutable. The type of field. }, ], "partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity. "userManaged": True or False, # Required. Whether the schema is user-managed or managed by the service. - Set user_manage to false if you would like Dataplex to help you manage the schema. You will get the full service provided by Dataplex discovery, including new data discovery, schema inference and schema evolution. You can still provide input the schema of the entities, for example renaming a schema field, changing CSV or Json options if you think the discovered values are not as accurate. Dataplex will consider your input as the initial schema (as if they were produced by the previous discovery run), and will evolve schema or flag actions based on that. - Set user_manage to true if you would like to fully manage the entity schema by yourself. This is useful when you would like to manually specify the schema for a table. In this case, the schema defined by the user is guaranteed to be kept unchanged and would not be overwritten. But this also means Dataplex will not provide schema evolution management for you. Dataplex will still be able to manage partition registration (i.e., keeping the list of partitions up to date) when Dataplex discovery is turned on and user_managed is set to true. }, "system": "A String", # Required. Immutable. Identifies the storage system of the entity data. "type": "A String", # Required. Immutable. The type of entity. "updateTime": "A String", # Output only. The time when the entity was last updated. }
list(parent, filter=None, pageSize=None, pageToken=None, view=None, x__xgafv=None)
List metadata entities in a zone. Args: parent: string, Required. The resource name of the parent zone: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}. (required) filter: string, Optional. The following filter parameters can be added to the URL to limit the entities returned by the API: Entity ID: ?filter="id=entityID" Asset ID: ?filter="asset=assetID" Data path ?filter="data_path=gs://my-bucket" Is HIVE compatible: ?filter=”hive_compatible=true” Is BigQuery compatible: ?filter=”bigquery_compatible=true” pageSize: integer, Optional. Maximum number of entities to return. The service may return fewer than this value. If unspecified, 100 entities will be returned by default. The maximum value is 500; larger values will will be truncated to 500. pageToken: string, Optional. Page token received from a previous ListEntities call. Provide this to retrieve the subsequent page. When paginating, all other parameters provided to ListEntities must match the call that provided the page token. view: string, Required. Specify the entity view to make a partial list request. Allowed values ENTITY_VIEW_UNSPECIFIED - The default unset value. The API will default to the FULL view. TABLES - Only list table entities. FILESETS - Only list fileset entities. x__xgafv: string, V1 error format. Allowed values 1 - v1 error format 2 - v2 error format Returns: An object of the form: { # List metadata entities response. "entities": [ # Entities in the specified parent zone. { # Represents tables and fileset metadata contained within a zone. "asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset. "catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry. "compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with. "bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, "hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, }, "createTime": "A String", # Output only. The time when the entity was created. "dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id. "dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv. "description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters. "displayName": "A String", # Optional. Display name must be shorter than or equal to 63 characters. "etag": "A String", # Optional. The etag for this entity. Required for update and delete requests. Must match the server's etag. "format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery. "compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed. "csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data. "delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified. "headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0. "quote": "A String", # Optional. The character used to quote column values. Accepts '"' and '''. Defaults to '"' if unspecified. }, "format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type. "json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified. }, "mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: - application/x-parquet - application/x-avro - application/x-orc - application/x-tfrecord - application/json - application/{subtypes} - text/csv - text/ - image/{image subtype} - video/{video subtype} - audio/{audio subtype} }, "id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores. Must begin with a letter. "name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}. "schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response. "fields": [ # Optional. The sequence of fields describing data in table entities. { # Represents a column field within a table schema. "description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters. "fields": [ # Optional. Any nested field for complex types. # Object with schema name: GoogleCloudDataplexV1SchemaSchemaField ], "mode": "A String", # Required. Additional field semantics. "name": "A String", # Required. The name of the field. The maximum length is 767 characters. The name must begins with a letter and not contains : and .. "type": "A String", # Required. The type of field. }, ], "partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data. { # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. "name": "A String", # Required. Partition name is editable if only the partition style is not HIVE compatible. The maximum length allowed is 767 characters. "type": "A String", # Required. Immutable. The type of field. }, ], "partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity. "userManaged": True or False, # Required. Whether the schema is user-managed or managed by the service. - Set user_manage to false if you would like Dataplex to help you manage the schema. You will get the full service provided by Dataplex discovery, including new data discovery, schema inference and schema evolution. You can still provide input the schema of the entities, for example renaming a schema field, changing CSV or Json options if you think the discovered values are not as accurate. Dataplex will consider your input as the initial schema (as if they were produced by the previous discovery run), and will evolve schema or flag actions based on that. - Set user_manage to true if you would like to fully manage the entity schema by yourself. This is useful when you would like to manually specify the schema for a table. In this case, the schema defined by the user is guaranteed to be kept unchanged and would not be overwritten. But this also means Dataplex will not provide schema evolution management for you. Dataplex will still be able to manage partition registration (i.e., keeping the list of partitions up to date) when Dataplex discovery is turned on and user_managed is set to true. }, "system": "A String", # Required. Immutable. Identifies the storage system of the entity data. "type": "A String", # Required. Immutable. The type of entity. "updateTime": "A String", # Output only. The time when the entity was last updated. }, ], "nextPageToken": "A String", # Token to retrieve the next page of results, or empty if there are no remaining results in the list. }
list_next(previous_request, previous_response)
Retrieves the next page of results. Args: previous_request: The request for the previous page. (required) previous_response: The response from the request for the previous page. (required) Returns: A request object that you can call 'execute()' on to request the next page. Returns None if there are no more items in the collection.
update(name, body=None, validateOnly=None, x__xgafv=None)
Update a metadata entity. Only supports full resource update. Args: name: string, Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}. (required) body: object, The request body. The object takes the form of: { # Represents tables and fileset metadata contained within a zone. "asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset. "catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry. "compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with. "bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, "hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, }, "createTime": "A String", # Output only. The time when the entity was created. "dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id. "dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv. "description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters. "displayName": "A String", # Optional. Display name must be shorter than or equal to 63 characters. "etag": "A String", # Optional. The etag for this entity. Required for update and delete requests. Must match the server's etag. "format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery. "compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed. "csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data. "delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified. "headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0. "quote": "A String", # Optional. The character used to quote column values. Accepts '"' and '''. Defaults to '"' if unspecified. }, "format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type. "json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified. }, "mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: - application/x-parquet - application/x-avro - application/x-orc - application/x-tfrecord - application/json - application/{subtypes} - text/csv - text/ - image/{image subtype} - video/{video subtype} - audio/{audio subtype} }, "id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores. Must begin with a letter. "name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}. "schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response. "fields": [ # Optional. The sequence of fields describing data in table entities. { # Represents a column field within a table schema. "description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters. "fields": [ # Optional. Any nested field for complex types. # Object with schema name: GoogleCloudDataplexV1SchemaSchemaField ], "mode": "A String", # Required. Additional field semantics. "name": "A String", # Required. The name of the field. The maximum length is 767 characters. The name must begins with a letter and not contains : and .. "type": "A String", # Required. The type of field. }, ], "partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data. { # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. "name": "A String", # Required. Partition name is editable if only the partition style is not HIVE compatible. The maximum length allowed is 767 characters. "type": "A String", # Required. Immutable. The type of field. }, ], "partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity. "userManaged": True or False, # Required. Whether the schema is user-managed or managed by the service. - Set user_manage to false if you would like Dataplex to help you manage the schema. You will get the full service provided by Dataplex discovery, including new data discovery, schema inference and schema evolution. You can still provide input the schema of the entities, for example renaming a schema field, changing CSV or Json options if you think the discovered values are not as accurate. Dataplex will consider your input as the initial schema (as if they were produced by the previous discovery run), and will evolve schema or flag actions based on that. - Set user_manage to true if you would like to fully manage the entity schema by yourself. This is useful when you would like to manually specify the schema for a table. In this case, the schema defined by the user is guaranteed to be kept unchanged and would not be overwritten. But this also means Dataplex will not provide schema evolution management for you. Dataplex will still be able to manage partition registration (i.e., keeping the list of partitions up to date) when Dataplex discovery is turned on and user_managed is set to true. }, "system": "A String", # Required. Immutable. Identifies the storage system of the entity data. "type": "A String", # Required. Immutable. The type of entity. "updateTime": "A String", # Output only. The time when the entity was last updated. } validateOnly: boolean, Optional. Only validate the request, but do not perform mutations. The default is false. x__xgafv: string, V1 error format. Allowed values 1 - v1 error format 2 - v2 error format Returns: An object of the form: { # Represents tables and fileset metadata contained within a zone. "asset": "A String", # Required. Immutable. The ID of the asset associated with the storage location containing the entity data. The entity must be with in the same zone with the asset. "catalogEntry": "A String", # Output only. The name of the associated Data Catalog entry. "compatibility": { # Provides compatibility information for various metadata stores. # Output only. Metadata stores that the entity is compatible with. "bigquery": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with BigQuery. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, "hiveMetastore": { # Provides compatibility information for a specific metadata store. # Output only. Whether this entity is compatible with Hive Metastore. "compatible": True or False, # Output only. Whether the entity is compatible and can be represented in the metadata store. "reason": "A String", # Output only. Provides additional detail if the entity is incompatible with the metadata store. }, }, "createTime": "A String", # Output only. The time when the entity was created. "dataPath": "A String", # Required. Immutable. The storage path of the entity data. For Cloud Storage data, this is the fully-qualified path to the entity, such as gs://bucket/path/to/data. For BigQuery data, this is the name of the table resource, such as projects/project_id/datasets/dataset_id/tables/table_id. "dataPathPattern": "A String", # Optional. The set of items within the data path constituting the data in the entity, represented as a glob path. Example: gs://bucket/path/to/data/**/*.csv. "description": "A String", # Optional. User friendly longer description text. Must be shorter than or equal to 1024 characters. "displayName": "A String", # Optional. Display name must be shorter than or equal to 63 characters. "etag": "A String", # Optional. The etag for this entity. Required for update and delete requests. Must match the server's etag. "format": { # Describes the format of the data within its storage location. # Required. Identifies the storage format of the entity data. It does not apply to entities with data stored in BigQuery. "compressionFormat": "A String", # Optional. The compression type associated with the stored data. If unspecified, the data is uncompressed. "csv": { # Describes CSV and similar semi-structured data formats. # Optional. Additional information about CSV formatted data. "delimiter": "A String", # Optional. The delimiter used to separate values. Defaults to ','. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8", and "ISO-8859-1". Defaults to UTF-8 if unspecified. "headerRows": 42, # Optional. The number of rows to interpret as header rows that should be skipped when reading data rows. Defaults to 0. "quote": "A String", # Optional. The character used to quote column values. Accepts '"' and '''. Defaults to '"' if unspecified. }, "format": "A String", # Output only. The data format associated with the stored data, which represents content type values. The value is inferred from mime type. "json": { # Describes JSON data format. # Optional. Additional information about CSV formatted data. "encoding": "A String", # Optional. The character encoding of the data. Accepts "US-ASCII", "UTF-8" and "ISO-8859-1". Defaults to UTF-8 if not specified. }, "mimeType": "A String", # Required. The mime type descriptor for the data. Must match the pattern {type}/{subtype}. Supported values: - application/x-parquet - application/x-avro - application/x-orc - application/x-tfrecord - application/json - application/{subtypes} - text/csv - text/ - image/{image subtype} - video/{video subtype} - audio/{audio subtype} }, "id": "A String", # Required. A user-provided entity ID. It is mutable, and will be used as the published table name. Specifying a new ID in an update entity request will override the existing value. The ID must contain only letters (a-z, A-Z), numbers (0-9), and underscores. Must begin with a letter. "name": "A String", # Output only. The resource name of the entity, of the form: projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{id}. "schema": { # Schema information describing the structure and layout of the data. # Required. The description of the data structure and layout. The schema is not included in list responses. It is only included in SCHEMA and FULL entity views of a GetEntity response. "fields": [ # Optional. The sequence of fields describing data in table entities. { # Represents a column field within a table schema. "description": "A String", # Optional. User friendly field description. Must be less than or equal to 1024 characters. "fields": [ # Optional. Any nested field for complex types. # Object with schema name: GoogleCloudDataplexV1SchemaSchemaField ], "mode": "A String", # Required. Additional field semantics. "name": "A String", # Required. The name of the field. The maximum length is 767 characters. The name must begins with a letter and not contains : and .. "type": "A String", # Required. The type of field. }, ], "partitionFields": [ # Optional. The sequence of fields describing the partition structure in entities. If this field is empty, there are no partitions within the data. { # Represents a key field within the entity's partition structure. You could have up to 20 partition fields, but only the first 10 partitions have the filtering ability due to performance consideration. "name": "A String", # Required. Partition name is editable if only the partition style is not HIVE compatible. The maximum length allowed is 767 characters. "type": "A String", # Required. Immutable. The type of field. }, ], "partitionStyle": "A String", # Optional. The structure of paths containing partition data within the entity. "userManaged": True or False, # Required. Whether the schema is user-managed or managed by the service. - Set user_manage to false if you would like Dataplex to help you manage the schema. You will get the full service provided by Dataplex discovery, including new data discovery, schema inference and schema evolution. You can still provide input the schema of the entities, for example renaming a schema field, changing CSV or Json options if you think the discovered values are not as accurate. Dataplex will consider your input as the initial schema (as if they were produced by the previous discovery run), and will evolve schema or flag actions based on that. - Set user_manage to true if you would like to fully manage the entity schema by yourself. This is useful when you would like to manually specify the schema for a table. In this case, the schema defined by the user is guaranteed to be kept unchanged and would not be overwritten. But this also means Dataplex will not provide schema evolution management for you. Dataplex will still be able to manage partition registration (i.e., keeping the list of partitions up to date) when Dataplex discovery is turned on and user_managed is set to true. }, "system": "A String", # Required. Immutable. Identifies the storage system of the entity data. "type": "A String", # Required. Immutable. The type of entity. "updateTime": "A String", # Output only. The time when the entity was last updated. }