How to serve huge file over streaming rest?

Question

I have 2 Spring Web applications: Application1 and Application2. In Application1, I have an endpoint at "http://application1/getbigcsv" that uses streaming in order to serve a gigantic 150MB CSV file back to the user if they hit that URL.

I dont want users to hit Application1 directly, but hit Application2 instead. If I have the following method in my controller in Application2

@RequestMapping(value = "/large.csv", method = GET, produces = "text/csv") @ResponseStatus(value = HttpStatus.OK) public String streamLargeCSV() { // Make an HTTP Request to http://application1/getbigcsv // Return its response }

My worry is the above is not doing "streaming" whereas Application1 is doing streaming. Is there some way I can make sure that the application2 will be serving back the same data from application1's rest endpoint in a streaming fashion? Or is the method above actually returning things in a "Streaming" method already because Application1 is serving its endpoint as streaming?

approxiblue · Accepted Answer · 2015-09-05 23:47:04Z

First of all: you can but not with that method signature.

Unfortunately, you have not shown how you produce that CSV file in app1, whether this is truly streaming. Let's assume it is.

You signature will look like this:

@RequestMapping(value = "/large.csv", method = GET, produces = "text/csv") @ResponseStatus(value = HttpStatus.OK) public void streamLargeCSV(OutputStream out) { // Make an HTTP Request to http://application1/getbigcsv // Return its response }

Now we have to grab the input stream from app1 first. Use Apache HttpClient to get your HttpEntity. This entity has a writeTo(OutputStream) method which will receive your out parameter. It will block until all bytes are consumed/streamed. When you are done, free all HttpClient resources.

Complete code:

@RequestMapping(value = "/large.csv", method = GET, produces = "text/csv") @ResponseStatus(value = HttpStatus.OK) public void streamLargeCSV(OutputStream out) { // Make an HTTP Request to http://application1/getbigcsv CloseableHttpClient httpclient = HttpClients.createDefault(); HttpGet httpGet = new HttpGet("http://application1/getbigcsv"); CloseableHttpResponse response = httpclient.execute(httpGet); try { HttpEntity entity = response.getEntity(); // Return its response entity.writeTo(out); } finally { response.close(); } }

Here is my real world example. Start reading from "Interesting to say what I have achieved in particular with this:"

You can use try-with-resources on CloseableHttpResponse for less boilerplate.
@approxiblue That is true. Feel free to refit my code to Java 7.

szefuf · Accepted Answer · 2015-09-09 13:57:34Z

In java.ws.rs.core package you have classes: StreamingOutput and ResponseBuilder.

Not sure if it will help you, but you may try.

Example:

@Produces("application/octet-stream") public Response doThings () { ... StreamingOutput so; try { so = new StreamingOutput() { public void write(OutputStream output) { … } }; } catch (Exception e) { ... } ResponseBuilder response = Response.ok(so); response.header("Content-Type", ... + ";charset=utf-8"); return response.build(); }

Gokhan Celikkaya · Accepted Answer · 2017-12-29 14:24:18Z

Change your methods return type to ResponseEntity<?> and return as following:

@GetMapping("/download") public ResponseEntity<?> fetchActivities( @RequestParam("filename") String filename) { String string = "some large text" InputStream is = new ByteArrayInputStream(string.getBytest()); HttpHeaders headers = new HttpHeaders(); headers.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=large.txt"); headers.add(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_OCTET_STREAM_VALUE); return ResponseEntity.ok().headers(headers).body(new InputStreamResource(is)); }

No serializer found for class java.io.FileDescriptor and no properties discovered to create BeanSerializer

Collectives™ on Stack Overflow

How to serve huge file over streaming rest?

3 Answers 3

2 Comments

1 Comment

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

1 Comment

1 Comment

Related