Get Ready to Boost your Prepare for your Databricks-Certified-Professional-Data-Engineer Exam with 220 Questions
Use Free Databricks-Certified-Professional-Data-Engineer Exam Questions that Stimulates Actual EXAM
NEW QUESTION 33
A data engineer has ingested data from an external source into a PySpark DataFrame raw_df. They need to
briefly make this data available in SQL for a data analyst to perform a quality assurance check on the data.
Which of the following commands should the data engineer run to make this data available in SQL for only
the remainder of the Spark session?
- A. There is no way to share data between PySpark and SQL
- B. raw_df.saveAsTable("raw_df")
- C. raw_df.createOrReplaceTempView("raw_df")
- D. raw_df.createTable("raw_df")
- E. raw_df.write.save("raw_df")
Answer: C
NEW QUESTION 34
Which of the following describes a benefit of a data lakehouse that is unavailable in a traditional data
warehouse?
- A. A data lakehouse enables both batch and streaming analytics
- B. A data lakehouse utilizes proprietary storage formats for data
- C. A data lakehouse captures snapshots of data for version control purposes
- D. A data lakehouse provides a relational system of data management
- E. A data lakehouse couples storage and compute for complete control
Answer: A
NEW QUESTION 35
The team has decided to take advantage of table properties to identify a business owner for each table, which of the following table DDL syntax allows you to populate a table property identifying the business owner of a table CREATE TABLE inventory (id INT, units FLOAT)
- A. SET TBLPROPERTIES business_owner = 'supply chain'
CREATE TABLE inventory (id INT, units FLOAT) - B. CREATE TABLE inventory (id INT, units FLOAT)
SET TAG (business_owner = 'supply chain') - C. CREATE TABLE inventory (id INT, units FLOAT)
SET (business_owner = 'supply chain') - D. CREATE TABLE inventory (id INT, units FLOAT)
SET PROPERTY (business_owner = 'supply chain') - E. TBLPROPERTIES (business_owner = 'supply chain')
Answer: E
Explanation:
Explanation
CREATE TABLE inventory (id INT, units FLOAT) TBLPROPERTIES (business_owner = 'supply chain') Table properties and table options (Databricks SQL) | Databricks on AWS Alter table command can used to update the TBLPROPERTIES ALTER TABLE inventory SET TBLPROPERTIES(business_owner , 'operations')
NEW QUESTION 36
Create a sales database using the DBFS location 'dbfs:/mnt/delta/databases/sales.db/'
- A. CREATE DATABASE sales USING LOCATION 'dbfs:/mnt/delta/databases/sales.db/'
- B. The sales database can only be created in Delta lake
- C. CREATE DELTA DATABASE sales LOCATION 'dbfs:/mnt/delta/databases/sales.db/'
- D. CREATE DATABASE sales LOCATION 'dbfs:/mnt/delta/databases/sales.db/'
- E. CREATE DATABASE sales FORMAT DELTA LOCATION 'dbfs:/mnt/delta/databases/sales.db/''
Answer: B
Explanation:
Explanation
The answer is
CREATE DATABASE sales LOCATION 'dbfs:/mnt/delta/databases/sales.db/'
Note: with the introduction of the Unity catalog and three-layer namespace usage of SCHEMA and DATABASE is interchangeable
NEW QUESTION 37
How do you check the location of an existing schema in Delta Lake?
- A. Use Data explorer
- B. Run SQL command DESCRIBE SCHEMA EXTENDED schema_name
E Schemas are internally in-store external hive meta stores like MySQL or SQL Server - C. Check unity catalog UI
- D. Run SQL command SHOW LOCATION schema_name
Answer: B
Explanation:
Explanation
Here is an example of how it looks
Graphical user interface, text, application, email Description automatically generated
NEW QUESTION 38
You are asked to create a model to predict the total number of monthly subscribers for a specific magazine.
You are provided with 1 year's worth of subscription and payment data, user demographic data, and 10 years
worth of content of the magazine (articles and pictures). Which algorithm is the most appropriate for building
a predictive model for subscribers?
- A. Linear regression
- B. Decision trees
- C. TF-IDF
- D. Logistic regression
Answer: A
NEW QUESTION 39
The research team has put together a funnel analysis query to monitor the customer traffic on the e-commerce platform, the query takes about 30 mins to run on a small SQL endpoint cluster with max scaling set to 1 cluster. What steps can be taken to improve the performance of the query?
- A. They can increase the cluster size anywhere from X small to 3XL to review the per-formance and select the size that meets the required SLA.
- B. They can turn on the Serverless feature for the SQL endpoint.
- C. They can turn on the Serverless feature for the SQL endpoint and change the Spot In-stance Policy from
"Cost optimized" to "Reliability Optimized." - D. They can increase the maximum bound of the SQL endpoint's scaling range anywhere from between 1 to 100 to review the performance and select the size that meets the re-quired SLA.
- E. They can turn off the Auto Stop feature for the SQL endpoint to more than 30 mins.
Answer: A
Explanation:
Explanation
The answer is, They can increase the cluster size anywhere from 2X-Small to 4XL(Scale Up) to review the performance and select the size that meets your SLA. If you are trying to improve the performance of a single query at a time having additional memory, additional worker nodes mean that more tasks can run in a cluster which will improve the performance of that query.
The question is looking to test your ability to know how to scale a SQL Endpoint(SQL Warehouse) and you have to look for cue words or need to understand if the queries are running sequentially or concurrently. if the queries are running sequentially then scale up(Size of the cluster from 2X-Small to 4X-Large) if the queries are running concurrently or with more users then scale out(add more clusters).
SQL Endpoint(SQL Warehouse) Overview: (Please read all of the below points and the below diagram to understand )
1.A SQL Warehouse should have at least one cluster
2.A cluster comprises one driver node and one or many worker nodes
3.No of worker nodes in a cluster is determined by the size of the cluster (2X -Small ->1 worker, X-Small ->2 workers.... up to 4X-Large -> 128 workers) this is called Scale Up
4.A single cluster irrespective of cluster size(2X-Smal.. to ...4XLarge) can only run 10 queries at any given time if a user submits 20 queries all at once to a warehouse with 3X-Large cluster size and cluster scaling (min
1, max1) while 10 queries will start running the remaining 10 queries wait in a queue for these 10 to finish.
5.Increasing the Warehouse cluster size can improve the performance of a query, example if a query runs for 1 minute in a 2X-Small warehouse size, it may run in 30 Seconds if we change the warehouse size to X-Small.
this is due to 2X-Small has 1 worker node and X-Small has 2 worker nodes so the query has more tasks and runs faster (note: this is an ideal case example, the scalability of a query performance depends on many factors, it can not always be linear)
6.A warehouse can have more than one cluster this is called Scale Out. If a warehouse is configured with X-Small cluster size with cluster scaling(Min1, Max 2) Databricks spins up an additional cluster if it detects queries are waiting in the queue, If a warehouse is configured to run 2 clusters(Min1, Max 2), and let's say a user submits 20 queries, 10 queriers will start running and holds the remaining in the queue and databricks will automatically start the second cluster and starts redirecting the 10 queries waiting in the queue to the second cluster.
7.A single query will not span more than one cluster, once a query is submitted to a cluster it will remain in that cluster until the query execution finishes irrespective of how many clusters are available to scale.
Please review the below diagram to understand the above concepts:
Scale-up-> Increase the size of the SQL endpoint, change cluster size from 2X-Small to up to 4X-Large If you are trying to improve the performance of a single query having additional memory, additional worker nodes and cores will result in more tasks running in the cluster will ultimately improve the performance.
During the warehouse creation or after, you have the ability to change the warehouse size (2X-Small....to
...4XLarge) to improve query performance and the maximize scaling range to add more clusters on a SQL Endpoint(SQL Warehouse) scale-out if you are changing an existing warehouse you may have to restart the warehouse to make the changes effective.
NEW QUESTION 40
A data engineer is using a Databricks SQL query to monitor the performance of an ELT job. The ELT job is triggered by a specific number of input records being ready to process. The Databricks SQL query returns the number of minutes since the job's most recent runtime. Which of the following approaches can enable the data engineering team to be notified if the ELT job has not been run in an hour?
- A. They can set up an Alert for the query to notify them if the returned value is greater than 60.
- B. They can set up an Alert for the accompanying dashboard to notify them if the returned value is greater than 60.
- C. They can set up an Alert for the query to notify when the ELT job fails.
- D. This type of alert is not possible in Databricks
- E. They can set up an Alert for the accompanying dashboard to notify when it has not re-freshed in 60 minutes.
Answer: A
Explanation:
Explanation
The answer is, They can set up an Alert for the query to notify them if the returned value is greater than 60.
The important thing to note here is that alert can only be setup on query not on the dashboard, query can return a value, which is used if alert can be triggered.
NEW QUESTION 41
Which of the following data workloads will utilize a Bronze table as its destination?
- A. A job that ingests raw data from a streaming source into the Lakehouse
- B. A job that enriches data by parsing its timestamps into a human-readable format
- C. A job that aggregates cleaned data to create standard summary statistics
- D. A job that queries aggregated data to publish key insights into a dashboard
- E. A job that develops a feature set for a machine learning application
Answer: A
Explanation:
Explanation
The answer is A job that ingests raw data from a streaming source into the Lakehouse.
The ingested data from the raw streaming data source like Kafka is first stored in the Bronze layer as first destination before it is further optimized and stored in Silver.
Medallion Architecture - Databricks
Bronze Layer:
1. Raw copy of ingested data
2. Replaces traditional data lake
3. Provides efficient storage and querying of full, unprocessed history of data
4. No schema is applied at this layer
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
Purpose of each layer in medallion architecture
NEW QUESTION 42
Define an external SQL table by connecting to a local instance of an SQLite database using JDBC
- A. 1.CREATE TABLE users_jdbc
2.USING org.apache.spark.sql.jdbc.sqlite
3.OPTIONS (
4. url = "jdbc:/sqmple_db",
5. dbtable = "users"
6.) - B. 1.CREATE TABLE users_jdbc
2.USING SQL
3.URL = {server:"jdbc:/sqmple_db",dbtable: "users"} - C. 1.CREATE TABLE users_jdbc
2.USING org.apache.spark.sql.jdbc
3.OPTIONS (
4. url = "jdbc:sqlite:/sqmple_db",
5. dbtable = "users"
6.) - D. 1.CREATE TABLE users_jdbc
2.USING SQLITE
3.OPTIONS (
4. url = "jdbc:/sqmple_db",
5. dbtable = "users"
6.) - E. 1.CREATE TABLE users_jdbc
2.USING SQL
3.OPTIONS (
4. url = "jdbc:sqlite:/sqmple_db",
5. dbtable = "users"
6.)
Answer: A
Explanation:
Explanation
The answer is,
1.CREATE TABLE users_jdbc
2.USING org.apache.spark.sql.jdbc
3.OPTIONS (
4. url = "jdbc:sqlite:/sqmple_db",
5. dbtable = "users"
6.)
Databricks runtime currently supports connecting to a few flavors of SQL Database including SQL Server, My SQL, SQL Lite and Snowflake using JDBC.
1.CREATE TABLE <jdbcTable>
2.USING org.apache.spark.sql.jdbc or JDBC
3.OPTIONS (
4. url = "jdbc:<databaseServerType>://<jdbcHostname>:<jdbcPort>",
5. dbtable " = <jdbcDatabase>.atable",
6. user = "<jdbcUsername>",
7. password = "<jdbcPassword>"
8.)
For more detailed documentation
SQL databases using JDBC - Azure Databricks | Microsoft Docs
NEW QUESTION 43
Which of the following tool provides Data Access control, Access Audit, Data Lineage, and Data discovery?
- A. Unity Catalog
- B. Data Governance
- C. Lakehouse
- D. DELTA lake
- E. DELTA LIVE Pipelines
Answer: A
NEW QUESTION 44
The operations team is interested in monitoring the recently launched product, team wants to set up an email alert when the number of units sold increases by more than 10,000 units. They want to monitor this every 5 mins.
Fill in the below blanks to finish the steps we need to take
* Create ___ query that calculates total units sold
* Setup ____ with query on trigger condition Units Sold > 10,000
* Setup ____ to run every 5 mins
* Add destination ______
- A. Python, Job, SQL Cluster, email address
- B. SQL, Job, SQL Cluster, email address
- C. Python, Job, Refresh, email address
- D. SQL, Alert, Refresh, email address
- E. SQL, Job, Refresh, email address
Answer: D
Explanation:
Explanation
The answer is SQL, Alert, Refresh, email address
Here the steps from Databricks documentation,
Create an alert
Follow these steps to create an alert on a single column of a query.
1.Do one of the following:
*Click Create in the sidebar and select Alert.
*Click Alerts in the sidebar and click the + New Alert button.
2.Search for a target query.
Graphical user interface, text, application Description automatically generated
To alert on multiple columns, you need to modify your query. See Alert on multiple col-umns.
3.In the Trigger when field, configure the alert.
*The Value column drop-down controls which field of your query result is evaluated.
*The Condition drop-down controls the logical operation to be applied.
*The Threshold text input is compared against the Value column using the Condition you specify.
Note
If a target query returns multiple records, Databricks SQL alerts act on the first one. As you change the Value column setting, the current value of that field in the top row is shown beneath it.
4.In the When triggered, send notification field, select how many notifications are sent when your alert is triggered:
*Just once: Send a notification when the alert status changes from OK to TRIGGERED.
*Each time alert is evaluated: Send a notification whenever the alert status is TRIGGERED regardless of its status at the previous evaluation.
*At most every: Send a notification whenever the alert status is TRIGGERED at a spe-cific interval. This choice lets you avoid notification spam for alerts that trigger of-ten.
Regardless of which notification setting you choose, you receive a notification whenever the status goes from OK to TRIGGERED or from TRIGGERED to OK. The schedule settings affect how many notifications you will receive if the status remains TRIGGERED from one execution to the next. For details, see Notification frequency.
5.In the Template drop-down, choose a template:
*Use default template: Alert notification is a message with links to the Alert configuration screen and the Query screen.
*Use custom template: Alert notification includes more specific information about the alert.
a.A box displays, consisting of input fields for subject and body. Any static content is valid, and you can incorporate built-in template variables:
*ALERT_STATUS: The evaluated alert status (string).
*ALERT_CONDITION: The alert condition operator (string).
*ALERT_THRESHOLD: The alert threshold (string or number).
*ALERT_NAME: The alert name (string).
*ALERT_URL: The alert page URL (string).
*QUERY_NAME: The associated query name (string).
*QUERY_URL: The associated query page URL (string).
*QUERY_RESULT_VALUE: The query result value (string or number).
*QUERY_RESULT_ROWS: The query result rows (value array).
*QUERY_RESULT_COLS: The query result columns (string array).
An example subject, for instance, could be: Alert "{{ALERT_NAME}}" changed status to
{{ALERT_STATUS}}.
b.Click the Preview toggle button to preview the rendered result.
Important
The preview is useful for verifying that template variables are rendered cor-rectly. It is not an accurate representation of the eventual notification content, as each alert destination can display notifications differently.
c.Click the Save Changes button.
6.In Refresh, set a refresh schedule. An alert's refresh schedule is independent of the query's refresh schedule.
*If the query is a Run as owner query, the query runs using the query owner's cre-dential on the alert's refresh schedule.
*If the query is a Run as viewer query, the query runs using the alert creator's cre-dential on the alert's refresh schedule.
7.Click Create Alert.
8.Choose an alert destination.
Important
If you skip this step you will not be notified when the alert is triggered.
NEW QUESTION 45
Which of the following locations hosts the driver and worker nodes of a Databricks-managed clus-ter?
- A. Control plane
- B. Databricks Filesystem
- C. Data plane
- D. Databricks web application
- E. JDBC data source
Answer: C
Explanation:
Explanation
See the Databricks high-level architecture
NEW QUESTION 46
What is the top-level object in unity catalog?
- A. Table
- B. Workspace
- C. Metastore
- D. Catalog
- E. Database
Answer: C
Explanation:
Explanation
Key concepts - Azure Databricks | Microsoft Docs
Diagram Description automatically generated
NEW QUESTION 47
you are currently working on creating a spark stream process to read and write in for a one-time micro batch, and also rewrite the existing target table, fill in the blanks to complete the below command sucesfully.
1.spark.table("source_table")
2..writeStream
3..option("____", "dbfs:/location/silver")
4..outputMode("____")
5..trigger(Once=____)
6..table("target_table")
- A. targetlocation, overwrite, True
- B. checkpointlocation, complete, True
- C. checkpointlocation, True, overwrite
- D. checkpointlocation, overwrite, True
- E. checkpointlocation, True, complete
Answer: B
NEW QUESTION 48
One of the team members Steve who has the ability to create views, created a new view called re-gional_sales_vw on the existing table called sales which is owned by John, and the second team member Kevin who works with regional sales managers wanted to query the data in region-al_sales_vw, so Steve granted the permission to Kevin using command GRANT VIEW, USAGE ON regional_sales_vw to [email protected] but Kevin is still unable to access the view?
- A. Steve is not the owner of the sales table
- B. Kevin is not the owner of the sales table
- C. Kevin needs select access on the table sales
- D. Kevin needs owner access on the view regional_sales_vw
- E. Table access control is not enabled on the table and view
Answer: A
Explanation:
Explanation
Ownership determines whether or not you can grant privileges on derived objects to other users, since Steve is not the owner of the underlying sales table, he can not grant access to the table or data in the table indirectly.
Only owner(user or group) can grant access to a object
https://docs.microsoft.com/en-us/azure/databricks/security/access-control/table-acls/object-privileges#a-user-has Data object privileges - Azure Databricks | Microsoft Doc
NEW QUESTION 49
The Delta Live Table Pipeline is configured to run in Production mode using the continuous Pipe-line Mode.
what is the expected outcome after clicking Start to update the pipeline?
- A. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will be deployed for the update and terminated when the pipeline is stopped
- B. All datasets will be updated at set intervals until the pipeline is shut down. The compute resources will persist after the pipeline is stopped to allow for additional testing
- C. All datasets will be updated once and the pipeline will shut down. The compute resources will be terminated
- D. All datasets will be updated continuously and the pipeline will not shut down. The compute resources will persist with the pipeline (Correct)
- E. All datasets will be updated once and the pipeline will shut down. The compute resources will persist to allow for additional testing
Answer: D
Explanation:
Explanation
The answer is,
All datasets will be updated continuously and the pipeline will not shut down. The compute re-sources will persist with the pipeline until it is shut down since the execution mode is chosen to be continuous. It does not matter if the pipeline mode is development or production, pipeline mode only matters during the pipeline initialization.
DLT pipeline supports two modes Development and Production, you can switch between the two based on the stage of your development and deployment lifecycle.
Development and production modes
Development:
When you run your pipeline in development mode, the Delta Live Tables system:
*Reuses a cluster to avoid the overhead of restarts.
*Disables pipeline retries so you can immediately detect and fix errors.
Production:
In production mode, the Delta Live Tables system:
*Restarts the cluster for specific recoverable errors, including memory leaks and stale cre-dentials.
*Retries execution in the event of specific errors, for example, a failure to start a cluster.
Use the buttons in the Pipelines UI to switch between develop-ment and production modes. By default,
pipelines run in development mode.
Switching between development and production modes only controls cluster and pipeline execution behavior.
Storage locations must be configured as part of pipeline settings and are not affected when switching between modes.
Delta Live Tables supports two different modes of execution:
Triggered pipelines update each table with whatever data is currently available and then stop the cluster running the pipeline. Delta Live Tables automatically analyzes the dependencies between your tables and starts by computing those that read from external sources. Tables within the pipe-line are updated after their dependent data sources have been updated.
Continuous pipelines update tables continuously as input data changes. Once an update is started, it continues to run until manually stopped. Continuous pipelines require an always-running cluster but ensure that downstream consumers have the most up-to-date data Please review additional DLT concepts using the below link
https://docs.databricks.com/data-engineering/delta-live-tables/delta-live-tables-concepts.html#delta-live-tables-c
NEW QUESTION 50
......
BEST Verified Databricks Databricks-Certified-Professional-Data-Engineer Exam Questions (2023) : https://www.exam4pdf.com/Databricks-Certified-Professional-Data-Engineer-dumps-torrent.html
Get 100% Real Databricks-Certified-Professional-Data-Engineer Free Online Practice Test: https://drive.google.com/open?id=1kEgHkvE6WuN5OhKLWJsx-hrTZbVRty2D
