0

I am having file which contain data like below:

100|hyd|xxx|32 101|chn|yyy|98 103|chn|abc|87 104|hyd|nbx|56

Here I want to filter the data based on location(hyd,chn) and store it in a text file. I tried the below code.

val file=sc.textFile("/home/cloudera/abc.txt") val file2=file.map(line=>line.split("\\|")) val file3 = file2.filter(line=>line.apply(1).matches("hyd")).saveAsTextFile("/home/cloudera/hyd")

When I check the /home/cloudera/hyd/part-00000 path data is stored in object format. [Ljava.lang.String;@679e1175

I want the data to be stored in plain text format.

100|hyd|xxx|32 104|hyd|nbx|56

Thank you.

Improve this question

1 Answer 1

Reset to default
2

You are just missing one thing converting the list to String!

This can be easily done in this way:

val file=sc.textFile("/home/cloudera/abc.txt") val file2=file.map(line=>line.split("\\|")) val file3 = file2.filter(line=>line.apply(1).matches("hyd")).map(line=>line.mkString("|")).saveAsTextFile("/home/cloudera/hyd")
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.