Menu Close

Some fun with Twitter #stats #dataanalytics #useracquisition #sideproject

Have you ever wondered how many usernames are still available on Twitter? How hard would it be to find a unique username on a social network which has been created a decade ago I do!

In this article I will try to demonstrate why these questions are more complex than it appeared at first glance.

Before we start, let’s have a look at the Twitter name policy.

The first point to highlight here is that contrary to other social networks Twitter does not free blocked accounts (unless you are a brand whose name has been taken). This means that all the accounts that have been blocked since 2006 are still locked!

While on the subject of Twitter policy, I also looked for some official stats concerning how many users registered since the tool was first released. Unfortunately, I was not able to find anything on the topic. Twitter, as many other tech companies, is keeping these numbers secret.

So without any official source of data how are we going to tackle this problem?

Let’s build our own tools!

My first idea was to design a bot that will take advantage of the built-in username checker. Indeed, who better than Twitter knows which names are still valid?

aaeaaqaaaaaaaajoaaaajduznjq1njmwlwvindctngi0zc1hntg3ltm5mzm5ymrlzjzmma

Without going into the technical details, I tried to hijack the API… And it was not a viable solution. Long story short: Twitter kicked me and I was not allowed to access my account anymore (oops).

Well, fair enough. Let’s try something else. Knowing that you don’t need to be logged in to go on a twitter profile, what about creating a web scrapper that will visit Twitter profiles and collect all the data for me?

I started by selecting a sample of usernames I found relevant: 2035 first names, which were given in France between 2004 and 2012.

According to you, how many are still available on Twitter?

To my great surprise, there was actually not a single one available anymore! Likewise, I tried with @_username and @username_ (as it’s advised in the Twitter name policy).

Once again, the percentage of available names today is under the 2%!

In search of an explanation

These surprisingly low results arose my curiosity. Thus, considering that the « join date » is displayed on the public profile how about trying to see when these username were taken?

I reconfigured my bot to get this new data and display the result in a chart. Here is the D3 representation I got:

See the Pen Twitter growth (classic FR names) by Adrien Rahier (@FracArt) on CodePen.0

(Dataset : 2035 French names)

Two statements can be made from this chart:

  • It seems that there was a huge peak of Twitter registration in the beginning of 2007.
  • Interestingly, we can see that every March a recurrent pattern occurred as new users are joining!

Concerning the first statement I tried to correlate these data to another source of information. According to a researched paperThe Tweets They are a-Changin’: Evolution of Twitter Users and Behavior, the peak of user acquisition as been reached in 2009 for Twitter. So, is my original sample (2035 French names) biased? Maybe.

Another hypothesis is that this peak on my chart shows the moment when there has been a popular interest for Twitter. Knowing that popular interest and massive user acquisition are two different concepts that might explained the gap.

Concerning the second statement I have to say that it’s completely unexpected. Does that mean that Twitter is actually releasing some usernames every March?

I ran some other tests to see if the same behavior was observable with some other data samples.

See the Pen Twitter growth (FR names with underscore ex: @_adrien) by Adrien Rahier (@FracArt) on CodePen.0

See the Pen Twitter growth (FR names with underscore ex: @adrien_) by Adrien Rahier (@FracArt) on CodePen.0

See the Pen Twitter growth (FR animals) by Adrien Rahier (@FracArt) on CodePen.0

(Datasets: 1- 2035 French names with underscore at the beginning, 2- 2035 French names with underscore at the end, 3- 1831 animals words)

Concerning the first statement, we still see on these three charts two peaks: one in 2007 and one in 2009 (yes!). However, the weird March pattern is less pronounced here.

Finally, as it as been suggested by a friend: Maybe some people are just doing some domain name squatting (the fact to reserve a domain name without doing anything to resell it later).

 To clarify this, I made a quick Excel bar chart to show the repartition of tweets per user.

aaeaaqaaaaaaaajmaaaajdrkzmvhnzqyltg5mzmtnddloc1hzdfhlty2zgvinjgzmjewnw

(Dataset : 2035 French names)

Once again, it’s hard to draw general conclusions with just one batch of users. However, it’s sounds like most of the accounts have at least tweeted two times.

Conclusion

To conclude this article and answer the original questions: Nowadays it’s going to be very hard to find a username that hasn’t been taken yet on Twitter.

For example, among the 1831 French animals word I tested, only the common terms @fourmilier and @vampire are still available!

Beyond this simple statement, this side project also showed me that it’s not very complicated to create and train a bot that will crawl some webpages for you.

Oddly enough, this market sector, automation of tasks to gather data, is still not claimed by any companies or startups. As far as I know the only example I can cite is the « Le camping » startup Phantombuster (read more about them here).

So there is probably a lot to do on this business segment. Indeed, from a business intelligence perspective this kind of informations can be very profitable (aka your company has access to some informations that your other competitors won’t have).

Finally, if you have some other ideas about how to interpret these charts or extend this side-project feel free to comment this article or ping me on Twitter : @AdrienRahier.

Additional notes:

  • All the code that has been used in this article is available on this Github repo.
  • You can see the four charts on my Codepen.
  • CasperJS has been used to scrap the data.
  • D3.js and Excel have been used to display the datas.

 

Confiance, politique et tasse de thé

Et l’Histoire se répéta.

Après le référendum européen en 2005 en France, c’est une fois de plus la victoire du « Non » qui a éclatée jeudi dernier en Angleterre.

Pour être tout à fait honnête, jusqu’à la publication des résultats je n’ai jamais vraiment voulu croire à une telle éventualité. Comment le camp du Brexit avec sa cohorte d’arguments démagogues et trompeurs pouvait-il l’emporter?

Bref, force est de le constater : je me suis lourdement trompé!
C
e vote m’a d’ailleurs confirmé une chose : quel que soit le pays d’Europe, les gens ont peur de l’avenir.

Pourquoi je crois en nous, l’Europe.

Déjà replaçons un petit peu le contexte. Je suis né en 1990, n’ai pas connu le mur de Berlin et me rappelle vaguement avoir utilisé le franc à un moment donné.

Aujourd’hui ce que je constate c’est que quelque soit le pays d’Europe la classe politique est aux abois, une grosse partie de la population est complètement angoissée, déboussolée, perdue … et surtout, surtout il y a une réelle défiance vis à vis des institutions politiques et financières.

Comme un certain nombre de jeunes de ma génération (il est vrai, parmi les plus favorisé) j’ai pourtant bénéficié d’énormes avantages suite à cette intégration Européenne :

  • La possibilité de voyager, d’aller voir ailleurs si bon me semble. Je me trouve aujourd’hui aux Pays-bas, cependant si demain l’envie m’en prenait je pourrais aller travailler en Belgique, Espagne, Allemagne.
  • Culturellement j’ai eu la chance de rencontrer plein de personnes dans pleins de pays (inter ou extra Europe) grâce à une langue commune : l’Anglais!
  • Les échanges avec d’autres universités m’ont été facilités (Eramus inside).
  • Quand un de nos gouvernements fait passer une loi limite niveau législatif, il y a toujours un recours devant la cours européenne possible.
  • L’Euro! La majorité des pays a adopté une monnaie compétitive, forte et surtout qui nous évite d’avoir à en changer à chaque fois que l’on passe une frontière.

Plus généralement si l’on y pense il y a en fait énormément de détails de nos vies quotidiennes qui nous semblent aujourd’hui naturels mais qui ne l’étaient pas il y a vingt ans.

Ce qu’il faudrait changer pour que cela marche

Le référendum au Royaume-Unis l’a montré. L’Union Européenne est loin de faire l’unanimité parmi ses citoyens.

Etant citoyen français, je vois les choses évidemment sous un prisme un petit peu différent que nos voisins. Après réflexion voici néanmoins quelques propositions sur lesquelles plancher de manière à améliorer ce sentiment d’appartenance à l’Union Européenne.

Jusqu’à présent en effet jamais je n’ai eu l’impression d’appartenir à l’UE (et pourtant je suis un pro-européen convaincu, vous imaginez pour les autres …). En outre, c’est quand même fou de se dire que jamais je n’ai été fier de dire à quelqu’un « Oui mais c’est parce que je suis Européen #tuvois ».
Beaucoup de français (comme c’est d’ailleurs le cas au RU) vantent leur indépendance, leur patriotisme. Pourquoi le pendant n’est-il jamais le même avec l’UE ?

Voici donc plusieurs idées concrètes pour penser global et non plus par pays :

  • Au 20h, à la télévision, ne plus se focaliser sur l’info France mais mettre l’accent sur l’international, l’Europe. Intégrer des chaînes en anglais aussi serait une bonne idée!
  • Organiser des opérations de communications efficaces pour promouvoir les autres pays (ex service civique dans un autre pays de l’UE, aide financière pour les personnes à petit revenu souhaitant travailler dans un autre pays). Vous noterez que mot efficace est ici important.
    Cela fonctionne bien avec d’autres sujets d’ordre public (certes un peu différents) comme le tabagisme pourquoi cela serait-il différent avec l’Europe?
  • Rendre la journée du 9 Mai férié (car oui le 9 Mai figurez-vous que c’est la journée de l’Europe) et utilisez ce tremplin pour en faire la promotion.
  • Lancer plus de sites au niveau européen. Je bosse dans l’informatique je suis donc bien placé pour savoir qu’une transition d’un site à portée nationale vers une globalité est sûrement une des choses les plus compliquer à réaliser. Cependant rien n’est impossible quand les financements suivent. Allez je lance une idée : Pourquoi ne pas ainsi construire un site qui vous mettrait en relation avec d’autres européens qui partagent les mêmes centre d’intérêt que vous ?

L’objectif est donc de faire des gestes forts, qu’enfin l’on mette les bouchées doubles sur ce sujet important. A plus long terme monsieur madame tout le monde devraient non plus voir les autres pays comme des menaces mais au contraire comme des alliés!

Evidemment la question qui va très vite arriver est « Qui va financer tout ça? ». Réponse simple : vous moi et tous les autres. Aujourd’hui le sujet est trop grave pour se voiler la face : nous n’y arriverons pas si nous ne nous en donnons pas les moyens collectivement.

Réfléchissez-y deux minutes, le jeu n’en vaut-il pas la chandelle ? Mettons que cet espace n’est jamais existé et l’on vous en présente le projet : Une association de 28 pays (!) avec un monnaie commune, des envies communes et la volonté d’instaurer la paix dans le monde.
Vous y croiriez ? Non, et bien pourtant cela existe, c’est réel et concret.

Freelance chronicles – Exploring a new world

It has been almost six months now since I started a freelance business. Looking back, I realize that it has been quite a while since I published something.  So let’s step back a little bit and draw the first conclusions.

Client prospection: what works vs. what doesn’t

When I launched my business in December, one of my first decisions had been to list and contact all the headhunters I knew.
Indeed, from my past experiences meeting this kind of people had always been synonym of success. So why not giving it a try one more time?

Well, I have to say that it didn’t take me long to realize that this approach was completely inefficient.
First of all I was not the right target. What a headhunter is looking for in priority is a topnotch employee… not a beginning freelancer. On the second hand, the niche market of Freelancing headhunters firm is mainly concerned about placing freelancers at the client offices directly. Once again, not a good fit for me (as I was only interested in remote missions).

As we are talking about disillusions, let’s now see the Hopwork case.
I heard about this platform approximately two years ago. At that moment they just raised some seed funds money and were (supposedly) the freelance market place that would disrupt the traditional labor market.
Well, what is obvious to me now is that beyond the marketing pitch this type of community websites do not work (but don’t get me wrong maybe I was not doing the right things either ;-)).

Fortunately despite my Hopwork misadventure I also had positive experiences with many other websites. Let’s look at three of them.

On the third place of this top: RemixJobs. This French generalist job platform is very well designed: when you select the right filters it sends you an email as soon as there is a match with your needs.
Secondly I would like to give a special mention to AngelList. Thank to this website I reach out a bunch of interesting prospects.

Finally on the top of the podium, Linkedin was my best business provider!
At the beginning of January I registered to a premium membership and added hundreds and hundreds of people. When you think about it: 1 Linkedin contact request = approximately 1 view on your profile … so therefore, more chance to get known. And it truly worked ;-).

My feedback: What were the keys to success?

1- At the beginning, clearly define your target

Before embarking headlong into the mass mailing campaign be sure to take a step back and analyze who is your preferred audience.
In my case I identified two types of customers who were the perfect fit for my profile: the fast-growing startups and the small/medium-size agencies.
Having worked previously within these two kinds of companies was giving me a decisive advantage: I knew already what their needs and pain points were.

2- Be sure to always have some projects to show to your clients

 Logically, when you start a freelance business you don’t have any client references yet. So to convince a new one that you are THE right person to hire a very good thing to do is to show your side-projects (having peer-references helps too).
So if you don’t have any yet, do not wait any longer and start coding!

 3- Winning a contract with a new customer: 5% chance + 95% of motivation and perseverance

Another important thing I noticed: To win a contract with a new customer you have to be “a finisher”.
It’s not only important to establish an interesting contact during the prospection phase it’s also incredibly critical to pay attention to all the details on the way to close a deal.

4- For every new client always start with a “trial period”

As an independent worker you are meeting and dealing with a lot of different people. The risk of not having the same needs, way of working or ambition is very high. So it’s really important to backup yourself (even when this client is a friend of a friend).
Fact: since I started my business, I had to give up the project after this trial period twice already In one case the customer was not paying me, and in the other the working conditions were not compatible with my other clients.

Speaking of this topic, be sure that things are clear on both sides before starting. If the trial period did not turn out well (for any reason) just simply quit the project.

5- Regularly step back and judge (objectively) your work

As a freelancer “you are your own company” so it’s relatively easy to lose control. Reviewing your results compared to the objective on a regular basis is a must!
Concretely at the end of each month I have a point in my agenda to detail what works and what did not over the past few weeks.

6- “Ménager sa monture”

 I really like this French expression. It basically means that if you are not taking care of your vehicle (aka yourself) you will have some problems later.
As an independent worker you don’t have this eight to five job anymore. As a consequence, is very easy to feel overworked and overpressured. So, to avoid that, save some time for yourself and don’t feel guilty to spend some free time outside of work (yeah, I am still working on it).

Conclusion:

As you can see the life of Freelancer is all except a “Long Quiet River” ;-). However I have to say that I don’t regret anything. Being an independent worker brings a lot of constraints but also a ton of flexibility in terms of organization, project management etc.

I hope you enjoyed reading this article, if so be sure to spread it over the Internet 😉

 You can also follow the rest of my adventures on Twitter: @AdrienRahier

Newer Posts
Older Posts