7.9k members in the apachespark community. Articles and discussion regarding anything to do with Apache Spark.
pyspark-concatenate-two-dataframes-row-wise
May 13, 2016 — The number of columns in each dataframe can be different. from pyspark.sql.functions import monotonically_increasing_id. df1 = sqlContext.. If you do not want to join, but rather combine the two into a single dataframe, you could use. df1.union(df2). To use union both dataframes should have the same .... how to do multiplication of two pyspark dataframe row wise ... Answer. You can do a cross join and multiply the columns using a list comprehension: result = df1. 939c2ea5af
pyspark concatenate two dataframes row wise, pyspark concatenate two dataframes column wise, pyspark merge dataframes column wise, pyspark merge dataframes row wise