mck
2020-12-09 02:32:42
Divide your column by 1000 and use F.from_unixtime to convert to timestamp type:
import pyspark.sql.functions as F
for d in dateFields:
df = df.withColumn(d,
(checkpoint / F.lit(1000.)).cast('timestamp')
)
热门帖子
热门github
6
It is working thank you, but i loosed the precision ->
17/7/2020 8:59. My wanted result should be like this17/7/2020 8:59:50.917.@Jonito see if edited answer helps?
Perfect thanks a lot ! Do you recommand me a good website or course where i can learn fast ?
read the docs: spark.apache.org/docs/latest/api/python/pyspark.sql.html