Skip to main content

Module utils

Module utils 

Source

StructsΒ§

FragmentDesc
PartialActorLocation
PartialFragmentStateTables
PartialObject
PartialUserPrivilege
StreamingJobExtraInfo

FunctionsΒ§

build_object_group_for_delete πŸ”’
Build a object group for notifying the deletion of the given objects.
build_select_node_list
check_connection_name_duplicate
check_connection_name_duplicate checks whether the connection name is already used in the target namespace.
check_database_name_duplicate
check_database_name_duplicate checks whether the database name is already used in the cluster.
check_function_signature_duplicate
check_function_signature_duplicate checks whether the function name and its signature is already used in the target namespace.
check_if_belongs_to_iceberg_table
check_relation_name_duplicate
check_relation_name_duplicate checks whether the relation name is already used in the target namespace.
check_schema_name_duplicate
check_schema_name_duplicate checks whether the schema name is already used in the target database.
check_secret_name_duplicate
check_sink_into_table_cycle
Check if create a sink with given dependent objects into the target table will cause a cycle, return true if it will.
check_subscription_name_duplicate
check_user_name_duplicate
check_user_name_duplicate checks whether the user is already existed in the cluster.
compose_dispatchers
construct_obj_dependency_query
This function will construct a query using recursive cte to find all objects[(id, obj_type)] that are used by the given object.
construct_privilege_dependency_query
construct_privilege_dependency_query constructs a query to find all privileges that are dependent on the given one.
construct_sink_cycle_check_query
This function will construct a query using recursive cte to find if dependent objects are already relying on the target table.
ensure_job_not_canceled
ensure_object_id
ensure_object_id ensures the existence of target object in the cluster.
ensure_privileges_not_referred
ensure_privileges_not_referred ensures that the privileges are not granted to any other users.
ensure_schema_empty
ensure_schema_empty ensures that the schema is empty, used by DROP SCHEMA.
ensure_user_id
ensure_user_id ensures the existence of target user in the cluster.
extract_external_table_name_from_definition
extract_grant_obj_id
fetch_target_fragments
filter_workers_by_resource_group
find_dirty_iceberg_table_jobs
get_belong_objects
get_belong_objects_by_ids
Returns all objects that transitively belong to object_ids, excluding object_ids.
get_database_resource_group
get_existing_job_resource_group
get_fragments_for_jobs
For the given streaming jobs, returns
get_iceberg_related_object_ids
get_iceberg_related_object_ids returns the related object ids of the iceberg source, sink and internal tables.
get_incoming_sink_objects_for_target πŸ”’
get_index_state_tables_by_table_id
get_internal_tables_by_id
get_object_owner
get_object_owner returns the owner of the given object.
get_referring_objects
List all objects that are using the given one.
get_referring_objects_cascade
List all objects that are using the given one in a cascade way. It runs a recursive CTE to find all the dependencies.
get_referring_privileges_cascade
get_sink_fragment_by_ids
get_streaming_job_extra_info
get_table_columns
get_user_privilege
get_user_privilege returns the privileges of the given user.
grant_default_privileges_automatically
grant_default_privileges_automatically grants default privileges automatically for the given new object. It returns the list of user infos whose privileges are updated.
has_table_been_migrated
insert_fragment_relations
list_object_dependencies
List object dependencies with optional filtering of non-created streaming jobs.
list_object_dependencies_by_object_id
List object dependencies for the specified object id without filtering by job status.
list_object_dependencies_impl πŸ”’
Collect object dependencies with optional object id filtering, and optionally exclude non-created streaming jobs.
list_user_info_by_ids
list_user_info_by_ids lists all users’ info by their ids.
load_streaming_jobs_by_ids πŸ”’
Load streaming jobs by job ids.
rebuild_fragment_mapping
rename_relation
rename_relation renames the target relation and its definition, it commits the changes to the transaction and returns the updated relations and the old name.
rename_relation_refer
rename_relation_refer updates the definition of relations that refer to the target one, it commits the changes to the transaction and returns all the updated relations.
resolve_no_shuffle_actor_mapping
Resolve 1:1 actor mapping between two no-shuffle-connected fragments.
to_pb_object_type πŸ”’
upsert_user_privileges πŸ”’
Insert user privileges idempotently using the same key as idx_user_privilege_item. An existing privilege is preserved, except that its grant option is upgraded when requested.
validate_restrict_drop_and_collect_owned_objects
Validates a RESTRICT drop and returns referring objects that should be dropped together.
validate_subscription_deletion
Validate that subscription can be safely deleted, meeting any of the following conditions:

Type AliasesΒ§

StreamingJobExtraInfoRow πŸ”’
Tuple of (job_id, timezone, config_override, backfill_orders, refresh_interval_sec)