Fabric – How to rename a column name when using a Python Notebook
I thought it would be good to help others in terms of my learning journey when working with partner notebooks and Microsoft fabric.
In today’s blog post, I am going to show you how to rename a column. In my experience this came up because I had a column name which had a forward slash “/” in it which caused the loading of the data for the table to fail because this is a reserved character.
If you find this blog post interesting and would like to get more blogs on how to manipulate data using Python notebooks, please let me know in the comments section. I do have quite a few other ideas!
I think it is also good to know how to rename a column because there are some instances when column name is not what is exactly needed to be loaded into Lakehouse table.
In my example I loaded a CSV table as shown below.
As far as I am aware currently column names cannot have a space in them for the column name.
As per my example above I have a column called “Order Number”.
Below is the python code I used to then rename the column from “Order Number” to “OrderNumber”
#Rename Column so that it will load correctly, by removing the space from 'Order Number'
from pyspark.sql import SparkSession
# Rename the 'Order Number' column to 'OrderNumber'
df = df.withColumnRenamed("Order Number", "OrderNumber")
#Show the changes of the column rename
display(df)
Now when I ran the above code, I got the following result below with the column renamed.
Summary
In this blog post I have shown you how to rename the column name when working with a Python data frame.
Once again, if you find this blog post interesting and would like to get more blogs on how to manipulate data using Python notebooks, please let me know in the comments section. I do have quite a few other ideas!
Thanks for reading!
[…] Gilbert Quevauvilliers performs a rename: […]
Great tip, thank you! One comment, would it be possible to extend the code to add how to import the df dataframe back to new csv file that will be loaded to a table? or maybe to directly load df into a Lake House table. Thank you.
Hi Ilya
This certainly can be done. You can take the new dataframe and then write into your table. I do not have the code right now, but you should be able to find it with a search on the internet.