The "natural" way of referencing resources like images in a WPF project is your second option. You can use a relative URI to point to the image and WPF will lazy load it. You can reference resources in other assemblies using pack URI syntax.
Using Resources.resx will code-generate properties that loads resources when referenced. Resources can be strings, images, icons or a byte arrays. Using {x:Static} in XAML allows you to reference the static properties generated by the code-generator but often you will need a converter to convert the resource type into a type usable by WPF.
There is some support for localization using Resources.resx and if you want to provide a multi-lingual application you could store the translated strings in Resources.resx. However, WPF localization as described by Microsoft is not based on Resources.resx.
For images, the second option is much easier. For strings, the first option is probably easier but instead you could stay in XAML and create a ResourceDictionary.