How to structure complex objects with related fields which need to be initialized at the same time

I am implementing an API and I keep running into this issue, I think there is something wrong with my core design but I'm not sure what and I'm feeling overwhelmed by design principles.

Basically I'll have an object with a bunch of related fields. It is non-trivial to populate the fields with the proper info and relies on one or more external API calls using a client I pass in to the constructor, these API calls also provide information related to more than one field, thus it is desirable to populate many fields at the same time. I want to keep the constructor simple/fast and keep my object testable so I don't put any logic there, just assignments. However, what I end up doing is creating a single method to populate all the fields and calling this in all of my getters after a null check, i.e. lazily populating the object fields.

I think that this is bad because it violates the "fail-fast" principle, especially because I am using a client to call an external service, which may fail if, say, the client credentials are invalid. However, I am having trouble restructuring my code.

I've thought about extracting the client logic into a service/connector, ClothingConnector for example, however I'm not sure this would solve my problem as I still wouldn't want to call this in the constructor, and it would still be beneficial to populate many fields at once.

class Person { ClientToGetClothing clothingClient; Pants pants; Shirt shirt; Fabric shirtFabric; Fabric pantsFabric; public Person(ClientToGetClothing clothingClient) { this.clothingClient = clothingClient; } private void populateClothing() { PantsResponse pantsInfo = clothingClient.retrievePantsInfo(); this.pants = pantsInfo.getPants(); ShirtResponse shirtInfo = clothingClient.retrieveShirtInfo(); this.shirt = pantsInfo.getShirt(); // do some more things with my pants + shirt and assign results to more fields, calculate the fabric in this example } public Shirt getShirt() { if (shirt == null) { populateClothing; } return this.shirt; } // ... }