# Time shift#

Time shifts in irradiance data is common, e.g., due daylight savings, incorrect timezone, or drift in datalogger timing.

## Incorrect timezone#

First, let’s take an existing dataset and purposefully set an incorrect timezone in order to investigate how this can be detected.

```import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
import pvlib
# sphinx_gallery_thumbnail_number = 2

filename = '../notebooks/data/solar_irradiance_dtu_2019_extended.csv'
df = pd.read_csv(filename, index_col=0, parse_dates=True)

# Remove timezone information
df = df.tz_convert(None)

# Set incorrect timezone
df = df.tz_localize('Etc/GMT-4', ambiguous='NaT', nonexistent='NaT')

# Calculate solar position using pvlib
location = pvlib.location.Location(latitude=55.7906, longitude=12.5253)
solpos = location.get_solarposition(df.index)
df['zenith'] = solpos['apparent_zenith']

# Calculate Global Horizontal Irradiance (GHI) from Diffuse Horizontal
# Irradiance (DHI) and Direct Normal Irradiance (DNI) using the closure
# equation.
df['ghi_calc'] = df['dhi'] + df['dni']*np.cos(np.deg2rad(df['zenith']))

# Compare the calculated and measured GHI:
df.plot.scatter(x='ghi', y='ghi_calc', s=0.1, alpha=0.1, grid=True)
plt.show()
```

In the above plot the measured and calculated GHI should ideally lie on a stright line. Clearly this is not the case, but rather there are distinct oval features, which is a common indication that the timezone is incorrect.

## 2-D heat map visualization#

Another very useful way of detecting time offset is to visualize the irradiance data using a heat map with the hour of day on the y-axis and the date on the x-axis.

```# Calculate sunrise/sunset for the entire period
days = pd.date_range(df.index[0], df.index[-1])
sunrise_sunset = location.get_sun_rise_set_transit(days)

# Convert sunrise/sunset from datetime to decimal hours
sunrise_sunset['sunrise'] = sunrise_sunset['sunrise'].dt.hour + \
sunrise_sunset['sunrise'].dt.minute/60
sunrise_sunset['sunset'] = sunrise_sunset['sunset'].dt.hour + \
sunrise_sunset['sunset'].dt.minute/60

df['hourofday'] = df.index.hour + df.index.minute/60

# Create dataframe with rows corresponding to days and columns to hours
df_2d = df[['ghi']].set_index([df.index.date, df.hourofday]).unstack(level=0)

# Calculate the extents of the 2D plot [x_start, x_end, y_start, y_end]
xlims = mdates.date2num([df.index[0].date(), df.index[-1].date()])
extent = [xlims[0], xlims[1], 0, 24]

# Plot heat map
fig, ax = plt.subplots()
im = ax.imshow(df_2d['ghi'],  aspect='auto', origin='lower', cmap='jet',
extent=extent, vmin=0, vmax=1000)

# Plot sunrise and sunset
ax.plot(mdates.date2num(sunrise_sunset.index),
sunrise_sunset[['sunrise', 'sunset']].to_numpy(),
c='r', linestyle='--', lw=2)

# Add colorbar
cbar = fig.colorbar(im, ax=ax, orientation='vertical', pad=0.01, label='GHI [W/m\$^2\$]')

# Format plot
ax.set_xlim(xlims)
ax.set_yticks([0, 6, 12, 18, 24])
ax.set_ylabel('Time of day [h]')
ax.set_facecolor('grey')
ax.xaxis_date()
ax.xaxis.set_major_formatter(mdates.DateFormatter('%b'))

plt.show()
```

In the above heat map the red lines represents sunrise and sunset times. It can be noted that the non-zero irradiance values are not all contained within the daytime period (between sunrise and sunset). This, is a clear indication that there is a timezone offset. Shifts in time due to daylight savings can also be detected using this method.

Total running time of the script: ( 0 minutes 14.954 seconds)

Gallery generated by Sphinx-Gallery