100% Databricks Databricks-Certified-Data-Engineer-Professional Guaranteed Success With Testing Engine

Databricks Databricks-Certified-Data-Engineer-Professional Exam Overview:

Databricks Databricks-Certified-Data-Engineer-Professional Exam Syllabus Topics:

Databricks Certified Data Engineer Professional Sample Questions:

Question 1

A data engineer wants to refactor the following DLT code, which includes multiple table definitions with very similar code.

In an attempt to programmatically create these tables using a parameterized table definition, the data engineer writes the following code.

The pipeline runs an update with this refactored code, but generates a different DAG showing incorrect configuration values for these tables.
How can the data engineer fix this?

A. Load the configuration values for these tables from a separate file, located at a path provided by a pipeline parameter.
B. Wrap the loop inside another table definition, using generalized names and properties to replace with those from the inner table
C. Convert the list of configuration values to a dictionary of table settings, using different input the for loop.
D. Convert the list of configuration values to a dictionary of table settings, using table names as keys.

Question 2

A data engineer needs to implement column masking for a sensitive column in a Unity Catalog- managed table. The masking logic must dynamically check if users belong to specific groups defined in a separate table (group_access) that maps groups to allowed departments. Which approach should the engineer use to efficiently enforce this requirement?

A. Create a UDF that hardcodes allowed groups and apply it as a column mask.
B. Apply a column mask that references the group_access mapping table in its UDF.
C. Use a row filter to restrict access based on the user's group.
D. Create a view without selecting the sensitive column.

Question 3

A data engineer is using Structured Streaming to read in transaction data from a bronze Delta table. It was discovered that the data has quality issues where sometimes the transaction value is negative, and when that occurs, the rows need to be routed to a separate quarantine table. They have low latency requirements for the good data since it is used by downstream systems, but the bad data will only be analyzed periodically and has no production dependencies. The quarantine job needs to be implemented so that it cannot affect the production processes that depend on the good data, and the cost of the job needs to be minimized. How should the quarantine process be implemented in order to satisfy these requirements?

A. The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. A new boolean column called "quarantine" should be added to the dataframe, and its value should be set to true if the transaction value is less than 0 and false if the transaction value is greater than or equal to 0. Processing and storing all the data together will save costs.
B. The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing. Both should run as separate streams on the same cluster to minimize cost.
C. The existing streaming job for the good data should be updated to incorporate the quarantining of the bad data. Inside a foreachBatch function, the dataframe should be filtered so that records with a transaction value greater than or equal to 0 are written to the good data table and records with a transaction value less than 0 are written to a quarantine table. Try/Catch can be added around the writes in the foreachBatch function so that the stream can't fail.
D. The streaming job for the good data needs to be modified to filter out records with a transaction value less than 0 before writing, and should not share compute with other processes. The streaming job for the quarantine data needs to filter out records with a transaction value greater than or equal to 0 before writing, and should be implemented on a separate small cluster and only run once a day to minimize cost.

Question 4

A data engineer wants to join a stream of advertisement impressions (when an ad was shown) with another stream of user clicks on advertisements to correlate when impressions led to monetizable clicks.
In the code below, Impressions is a streaming DataFrame with a watermark ("event_time", "10 minutes")

The data engineer notices the query slowing down significantly.
Which solution would improve the performance?

A. Joining on event time constraint: clickTime == impressionTime using a leftOuter join
B. Joining on event time constraint: clickTime + 3 hours < impressionTime - 2 hours
C. Joining on event time constraint: clickTime >= impressionTime - interval 3 hours and removing watermarks
D. Joining on event time constraint: clickTime >= impressionTime AND clickTime <= impressionTime interval 1 hour

Question 5

A data ingestion task requires a one-TB JSON dataset to be written out to Parquet with a target part-file size of 512 MB. Because Parquet is being used instead of Delta Lake, built-in file-sizing features such as Auto-Optimize & Auto-Compaction cannot be used.
Which strategy will yield the best performance without shuffling data?

A. Set spark.sql.files.maxPartitionBytes to 512 MB, ingest the data, execute the narrow transformations, and then write to parquet.
B. Ingest the data, execute the narrow transformations, repartition to 2,048 partitions (1TB*
1024*1024/512), and then write to parquet.
C. Set spark.sql.shuffle.partitions to 512, ingest the data, execute the narrow transformations, and then write to parquet.
D. Set spark.sql.adaptive.advisoryPartitionSizeInBytes to 512 MB bytes, ingest the data, execute the narrow transformations, coalesce to 2,048 partitions (1TB*1024*1024/512), and then write to parquet.
E. Set spark.sql.shuffle.partitions to 2,048 partitions (1TB*1024*1024/512), ingest the data, execute the narrow transformations, optimize the data by sorting it (which automatically repartitions the data), and then write to parquet.

Solutions: