Free Preparation Discussions
MENU
Databricks Exam Databricks-Certified-Associate-Developer-for-Apache-Spark-3.0 Topic 2 Question 68 Discussion
Topic #: 2
The code block displayed below contains an error. The code block should write DataFrame transactionsDf as a parquet file to location filePath after partitioning it on column storeId. Find the error.
Code block:
transactionsDf.write.partitionOn("storeId").parquet(filePath)
No method partitionOn() exists for the DataFrame class, partitionBy() should be used instead.
Correct! Find out more about partitionBy() in the documentation (linked below).
The operator should use the mode() option to configure the DataFrameWriter so that it replaces any existing files at location filePath.
No. There is no information about whether files should be overwritten in the question.
The partitioning column as well as the file path should be passed to the write() method of DataFrame transactionsDf directly and not as appended commands as in the code block.
Incorrect. To write a DataFrame to disk, you need to work with a DataFrameWriter object which you get access to through the DataFrame.writer property - no parentheses involved.
Column storeId should be wrapped in a col() operator.
No, this is not necessary - the problem is in the partitionOn command (see above).
The partitionOn method should be called before the write method.
Wrong. First of all partitionOn is not a valid method of DataFrame. However, even assuming partitionOn would be replaced by partitionBy (which is a valid method), this method is a method of
DataFrameWriter and not of DataFrame. So, you would always have to first call DataFrame.write to get access to the DataFrameWriter object and afterwards call partitionBy.
More info: pyspark.sql.DataFrameWriter.partitionBy --- PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3, Question: 33 (Databricks import instructions)
Galen
2 months agoIvan
2 months agoJeannetta
27 days agoBobbye
1 months agoKristel
1 months agoDarnell
2 months agoRhea
2 months agoTwila
2 months agoEvangelina
1 months agoCarmela
1 months agoLai
1 months agoCaprice
2 months agoCristal
2 months agoTracie
2 months agoJohnson
2 months agoFelice
2 months agoTerrilyn
2 months agoRodrigo
2 months agoAlline
2 months agoTeddy
3 months agoCorazon
2 months agoShaniqua
2 months agoDelbert
2 months agoAlbert
3 months agoRene
3 months agoGeorgiana
3 months ago