Try Databricks : Certified-Data-Engineer-Professional valid & accurate questions and answers

A data engineering team needs to create a SQL Alert that monitors data quality across multiple columns in their customer table. They want to trigger an alert when both the percentage of customers with missing email addresses exceeds 15% AND the percentage of customers with invalid phone number formats exceeds 10%. Which SQL query pattern is appropriate for implementing this multi-column alert condition?

• A. SELECT CASE WHEN email_null_pct >15 AND phone_invalid_pct> 10 THEN 1 ELSE 0 END FROM (SELECT (COUNT (CASE WHEN email IS NULL THEN 1 END) * 100.0 / COUNT (*)) as phone_invalid_pct FROM customers) metrics
• B. SELECT email, phone FROM customers WHERE email IS NULL AND phone NOT RLIKE 'ˆ[0-9-
  +()\\s]+$'
• C. SELECT COUNT (*) FROM customers WHERE email IS NULL OR phone_format_invalid = true
• D. SELECT email_null_pct, phone_invalid_pct FROM (SELECT (COUNT(CASE WHEN email IS NULL THEN 1 END) *
  100.0/COUNT (*)) as email_null_pct, (COUNT(CASE WHEN phone NOT RLIKE 'ˆ[0-9-+()\\s]+$' THEN 1 END)*
  100.0/COUNT (*)) as phone_invalid_pct FROM customers)

A view is registered with the following code:

Both users and orders are Delta Lake tables.
Which statement describes the results of querying recent_orders?

• A. All logic will execute at query time and return the result of joining the valid versions of the source tables at the time the query began.
• B. All logic will execute when the table is defined and store the result of joining tables to the DBFS; this stored data will be returned when the table is queried.
• C. The versions of each source table will be stored in the table transaction log; query results will be saved to DBFS with each query.
• D. All logic will execute at query time and return the result of joining the valid versions of the source tables at the time the query finishes.

A Delta Lake table in the Lakehouse named customer_parsams is used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data engineering team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources.
Immediately after each update succeeds, the data engineer team would like to determine the difference between the new version and the previous of the table. Given the current implementation, which method can be used?

• A. Parse the Spark event logs to identify those rows that were updated, inserted, or deleted.
• B. Parse the Delta Lake transaction log to identify all newly written data files.
• C. Execute a query to calculate the difference between the new version and the previous version using Delta Lake's built-in versioning and time travel functionality.
• D. Execute DESCRIBE HISTORY customer_churn_params to obtain the full operation metrics for the update, including a log of all records that have been added or modified.

A data engineer is creating a daily reporting job. There are two reporting notebooks--one for weekdays and one for weekends. An "if/else condition" task is configured as
{{job.start_time.is_weekday}} == true to route the job to either the weekday or weekend notebook tasks. The same job would be used across multiple time zones. Which action should a senior data engineer take upon reviewing the job to merge or reject the pull request?

• A. Reject, as the {{job.start_time.is_weekday}} is for the UTC timezone.
• B. Merge, as the job configuration looks good.
• C. Reject, as they should use {{job.trigger_time.is_weekday}} instead.
• D. Reject, as the {{job.start_time.is_weekday}} is not a valid value reference.

The data engineer is using Spark's MEMORY_ONLY storage level. Which indicators should the data engineer look for in the spark UI's Storage tab to signal that a cached table is not performing optimally?

• A. On Heap Memory Usage is within 75% of off Heap Memory usage
• B. Size on Disk is < Size in Memory
• C. Size on Disk is> 0
• D. The RDD Block Name included the '' annotation signaling failure to cache
• E. The number of Cached Partitions> the number of Spark Partitions