El método plot de pandas tiene argumentos para agregar un título y etiquetas de los ejes directamente. Para tener más control sobre su aspecto utiliza los métodos de matplotlib de los ejes que devuelve plot.
Pasa el texto del título al argumento title.
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame({"Madrid": [6, 8, 11, 13, 17, 23, 26, 25, 21, 15, 10, 7],
"Londres": [5, 6, 8, 10, 13, 16, 18, 18, 15, 12, 8, 6],
"Oslo": [-4, -3, 1, 5, 11, 15, 17, 16, 11, 6, 1, -3]},
index = ["Ene", "Feb", "Mar", "Abr", "May", "Jun",
"Jul", "Ago", "Sep", "Oct", "Nov", "Dic"])
df.plot(title = "Temperatura media mensual")
# plt.show()

Por defecto pandas utiliza el nombre del índice como etiqueta del eje X. Establece las etiquetas de ambos ejes con xlabel e ylabel.
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame({"Madrid": [6, 8, 11, 13, 17, 23, 26, 25, 21, 15, 10, 7],
"Londres": [5, 6, 8, 10, 13, 16, 18, 18, 15, 12, 8, 6],
"Oslo": [-4, -3, 1, 5, 11, 15, 17, 16, 11, 6, 1, -3]},
index = ["Ene", "Feb", "Mar", "Abr", "May", "Jun",
"Jul", "Ago", "Sep", "Oct", "Nov", "Dic"])
df.plot(xlabel = "Mes", ylabel = "Temperatura (ºC)")
# plt.show()

Al utilizar subplots = True puedes pasar una lista a title, con un título para cada subplot.
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame({"Madrid": [6, 8, 11, 13, 17, 23, 26, 25, 21, 15, 10, 7],
"Londres": [5, 6, 8, 10, 13, 16, 18, 18, 15, 12, 8, 6],
"Oslo": [-4, -3, 1, 5, 11, 15, 17, 16, 11, 6, 1, -3]},
index = ["Ene", "Feb", "Mar", "Abr", "May", "Jun",
"Jul", "Ago", "Sep", "Oct", "Nov", "Dic"])
df.plot(subplots = True, layout = (3, 1), legend = False, sharey = True,
title = ["Madrid", "Londres", "Oslo"])
# plt.show()

plot devuelve unos ejes de matplotlib. Utiliza set_title, set_xlabel y set_ylabel para personalizar el tamaño de fuente, el color, la posición o el grosor de los textos.
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame({"Madrid": [6, 8, 11, 13, 17, 23, 26, 25, 21, 15, 10, 7],
"Londres": [5, 6, 8, 10, 13, 16, 18, 18, 15, 12, 8, 6],
"Oslo": [-4, -3, 1, 5, 11, 15, 17, 16, 11, 6, 1, -3]},
index = ["Ene", "Feb", "Mar", "Abr", "May", "Jun",
"Jul", "Ago", "Sep", "Oct", "Nov", "Dic"])
ax = df.plot(linewidth = 2)
ax.set_title("Temperatura media mensual", fontsize = 14,
loc = "left", fontweight = "bold", color = "#1D3557")
ax.set_ylabel("Temperatura (ºC)", color = "gray")
# plt.show()

Título de la figura
Para agregar un título general sobre varios subplots utiliza plt.suptitle.
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame({"Madrid": [6, 8, 11, 13, 17, 23, 26, 25, 21, 15, 10, 7],
"Londres": [5, 6, 8, 10, 13, 16, 18, 18, 15, 12, 8, 6],
"Oslo": [-4, -3, 1, 5, 11, 15, 17, 16, 11, 6, 1, -3]},
index = ["Ene", "Feb", "Mar", "Abr", "May", "Jun",
"Jul", "Ago", "Sep", "Oct", "Nov", "Dic"])
df.plot(subplots = True, layout = (3, 1), legend = False, sharey = True)
plt.suptitle("Temperatura por ciudad")
# plt.show()

Consulta el tutorial sobre los títulos en matplotlib para más opciones.
También te puede interesar