I would use a timestamp range and the intersection with the defined nightshift to calculate this.
select shift_id, start_datetime, end_datetime, upper(in_night) - lower(in_night) as time_during_night from ( select shift_id, start_datetime, end_datetime, tsrange(start_datetime, end_datetime) * tsrange(start_datetime::date + time '23:00', end_datetime::date + time '06:00') in_night from data ) t
tsrange(start_datetime, end_datetime) creates a timestamp range and tsrange(start_datetime::date + time '23:00', end_datetime::date + time '06:00') creates the coresponding range for those dates for the night shift.
The * operator then calculates the intersection between those two ranges. The outer query then simply subtracts the lower end of that range from the upper, yielding an interval that contains the duration during the night shift.
The derived table (sub-query) is only there, so that I don't have to repeat the expression for the ranges when doing the subtraction.
Online example