4
$\begingroup$

I'm trying to perform a hierarchical clustering, to aggregate some "zones" or neighborhoods of a city, based on the language that is used most in that zone

In order to do so, I have at hand a dataset provided by Twitter that gives me the coordinates (long and lat) where a tweet was posted and the language in which it is written

So I need an operative definition of distance between two languages!

Any ideas?

EDIT I actually found this:

https://alternativetransport.wordpress.com/2015/05/05/34/

It just doesn't come with a numerical table of distances, but I'll try to ask for one :)

Improve this question
$\endgroup$
4
  • $\begingroup$ I'm afraid that this question would be more on-topic on some kind of linguistic site rather than here... $\endgroup$ Commented May 23, 2016 at 10:24
  • $\begingroup$ Maybe what you need is not a distance between languages, but a classification method to identify the language from the twitter text. $\endgroup$ Commented May 23, 2016 at 10:52
  • $\begingroup$ Twitter already provides it! My task is to see if the neighborhoods that are in a city, actually represent the people that live there from a linguistical point of view $\endgroup$ Commented May 23, 2016 at 10:53
  • $\begingroup$ @Tim: I think a question on how to define & construct a distance metric - rather than on where to find one - would be on-topic, if perhaps a little broad. $\endgroup$ Commented May 23, 2016 at 11:06

1 Answer 1

Reset to default
3
$\begingroup$

The simplest option might be to ignore distances between languages themselves. You could construct a vector for each neighborhood, where each element corresponds to a language, and the its value corresponds to the number of tweets in that language. The distance function on these vectors would only take into account differences per element, but have no notion of the topology of the elements themselves (e.g. standard $l_2$/cosine/etc. distances).

Adding information about similarity between languages is interesting. Historical linguists have reconstructed trees specifying which languages derived from which others, including reconstructed historical languages like Proto-Indo-European. It's fun to consider measuring distance along these trees, but I have no idea if it would be practically useful.

Maybe it would be possible to do something like look at the extent to which the languages co-occur or are geographically clustered in other parts of the (modern) world. Two languages would be similar if they tend to both be spoken in some other country (or nearby countries).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you this is an option, but what if I want to see if the neighborhoods, that are set by the city hall, actually represent the people that live inside them from a linguistical point of view? $\endgroup$ Commented May 23, 2016 at 10:56
  • $\begingroup$ It seems like spoken language is a proxy for recent country or region of origin - but earlier immigration - or worse colonisation trends - could see languages that are not linguistically related spoken in the same location, such as in India or Singapore's choices of official languages. $\endgroup$ Commented May 23, 2016 at 11:10

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.