In my experience, these displays need at least 4-5V between the VCC pin and the contrast pin, sometimes more. Which means you need to either power them with 5V and adjust the contrast pin to 0..1V, or you can power them with 3.3V, but you'll need negative voltage on the contrast pin. The latter is preferred if your MCU is not 5V-tolerant. Perhaps your display needs a negative voltage even at 5V.
A similar display I've got actually has unpopulated pads for a MAX660 charge pump which generates reverse voltage for the contrast circuit: 
Depending on the matrix, it may also be impossible to see anything without backlight. Green displays with black characters are usually readable in ambient light, but black displays with blue or green characters need backlight to be readable.