Pages in topic:   [1 2] >
Default ranking in Translators Directory
Thread poster: Gabriella Vento
Gabriella Vento
Gabriella Vento  Identity Verified
United States
Local time: 01:42
Member (2015)
English to Hungarian
+ ...
Feb 25, 2020

The Directory is undoubtedly the most useful tool for outsourcers to find the right freelancers, and therefore have a great impact on the opportunities offered for each translator. The intricate, sophisticated Directory algorithm can serve this purpose well, except some anomalies that I would like to share here and hear the opinion on it from you all.

By default, the system ranks Translators of a given language pair by the number of their Kudos, as opposed by their number of WWA-s.
... See more
The Directory is undoubtedly the most useful tool for outsourcers to find the right freelancers, and therefore have a great impact on the opportunities offered for each translator. The intricate, sophisticated Directory algorithm can serve this purpose well, except some anomalies that I would like to share here and hear the opinion on it from you all.

By default, the system ranks Translators of a given language pair by the number of their Kudos, as opposed by their number of WWA-s. (Wanting to Work Again)

- Kudos show, how many times a colleague found your answer the best for a given terminology question. Kudo numbers are useful, but can be misused; e.g. friends can mutually bring each other up to a high Kudos number, ensuring prime positions for each other.

- The number of WWAs show, how many outsourcers want to work with the translator again. Due to the stricter rules, (max 2 unverified WWA, everything else has to come from a Proz member outsourcer/client) misusing it is nearly impossible, thus - besides of other factors to be considered - the number reflect the true quality of a translator. WWA-s are easily trackable and have a date, as opposed to the Kudos.
In the recent setup the number of WWAs have very limited role in the ranking system. Set the listing according to WWAs is possible, but - even within this option - the ranking distinction is limited to only: "no" WWA-s or "1 or more" WWA-s.
If "A" translator has 1 WWAs and "B" translator has 30 WWAs, then "B" has no advantage over "A" in positioning at all.

From what I heard, most PM-s and clients use the Directory as it is, in the default setting. Perhaps only a few seasoned HR professionals go really deeply into the numerous classification options that the system offers.

Based on the above I suggest to modify the algorithm to make the default list set by WWAs (and their actual numbers, while at it) instead of number of Kudos.

Edit: I should have been more precise up there. I meant the number of positive WWAs should define the ranks. Then again, there are no negative WWAs exist in this system, and beyond that only one rating seems to be allowed for each of us.





[Edited at 2020-02-26 05:39 GMT]
Collapse


Amin Zanganeh Inaloo
 
Vanda Nissen
Vanda Nissen  Identity Verified
Australia
Local time: 18:42
Member (2008)
English to Russian
+ ...
Agree and disagree Feb 26, 2020

Gabriela,

I see your point, and it might be a good idea to make WWA more visible, however I do not agree that they should prevail over Kudoz points. Technically, you can get 30 WWA entries in one month, but it is impossible to get a lot of Kudoz points straight away. Answering questions takes time and effort. In fact, it tells more about a translator than mere WWAs. This is an unbiased opinion because I have both.

Although Kudoz debate sometimes get heated, I have not
... See more
Gabriela,

I see your point, and it might be a good idea to make WWA more visible, however I do not agree that they should prevail over Kudoz points. Technically, you can get 30 WWA entries in one month, but it is impossible to get a lot of Kudoz points straight away. Answering questions takes time and effort. In fact, it tells more about a translator than mere WWAs. This is an unbiased opinion because I have both.

Although Kudoz debate sometimes get heated, I have not witnessed a single translator in my pairs voting for a friend. If you have witnessed such an unethical behaviour, you should certainly report it to your moderator.
Collapse


Natalia Potashnik
Tainá Ferreira
 
Katalin Horváth McClure
Katalin Horváth McClure  Identity Verified
United States
Local time: 04:42
Member (2002)
English to Hungarian
+ ...
The problem with looking at the number of WWA-s Feb 26, 2020

We all work with a different mix of clients. Some freelancers work with fewer number of established clients who are steady suppliers of work, often larger projects. Others, on the other hand, work with a lot of different clients on smaller jobs (think about official documents, certificates, etc.) Obviously, someone with the first type client portfolio won't be able to get as many WWAs as someone with a client portfolio of the second type, even though the quality of their work may be the same.... See more
We all work with a different mix of clients. Some freelancers work with fewer number of established clients who are steady suppliers of work, often larger projects. Others, on the other hand, work with a lot of different clients on smaller jobs (think about official documents, certificates, etc.) Obviously, someone with the first type client portfolio won't be able to get as many WWAs as someone with a client portfolio of the second type, even though the quality of their work may be the same.
Also, please don't forget that some clients do not participate in the WWA system at all, as they consider their working relationship with their contractors confidential. So, some freelancers are simply not able to get WWAs at all from their very satisfied clients.
That is why the number of WWAs should never be used as a measure of the quality of one's work.
The average WWA rating value may be a better measure.
Collapse


Gabriella Vento
Philip Lees
Katalin Szilárd
Maciek Drobka
ahartje
Maria Teresa Borges de Almeida
Helen Shiner
 
Samuel Murray
Samuel Murray  Identity Verified
Netherlands
Local time: 09:42
Member (2006)
English to Afrikaans
+ ...
Suggestion: rank by ranking points, not KudoZ points Feb 26, 2020

I understand the logic behind using KudoZ points as the main driver for directory ranking, even though some translators who are very good may not participate in KudoZ at all. KudoZ points on average show whether a translator's peers consider him to be of acceptable standard w.r.t. issues in a particular language. However, I agree that the way KudoZ points are used, needs to be rethought (*how* is a topic for another discussion).

WWAs don't really say anything about a translator's
... See more
I understand the logic behind using KudoZ points as the main driver for directory ranking, even though some translators who are very good may not participate in KudoZ at all. KudoZ points on average show whether a translator's peers consider him to be of acceptable standard w.r.t. issues in a particular language. However, I agree that the way KudoZ points are used, needs to be rethought (*how* is a topic for another discussion).

WWAs don't really say anything about a translator's ability in a language. WWAs are more about client satisfaction (and in some cases, not even client satisfaction but simply confirming that the translator is real). Since WWAs are not given by peers, it is a less reliable way to measure competence. Still, I believe it should affect directory ranking.

In fact, I believe other things should also affect directory ranking, but there is currently no easy, transparent way of affecting it.

So, I would propose this change:

Do not use KudoZ points for ranking, but rather use "ranking points" for ranking, in which 1 KudoZ point = 10 ranking points. Then, allow other things to count ranking points as well, e.g. 1 WWA = 50 ranking points, 1 peer testimonial = 100 ranking points, 1 award = +10% ranking boost for the duration of the award, 1 KudoZ point from the past year = an additional 5 ranking points for the duration of the year, etc. The ranking exchange rate will be somewhat arbitrary but it will disconnect the ranking from KudoZ while retaining the importance of KudoZ for ranking.
Collapse


Sheila Wilson
Morano El-Kholy
 
Samuel Murray
Samuel Murray  Identity Verified
Netherlands
Local time: 09:42
Member (2006)
English to Afrikaans
+ ...
@Katalin Feb 26, 2020

Katalin Horváth McClure wrote:
Someone with the first type of client portfolio won't be able to get as many WWAs as someone with a client portfolio of the second type, even though the quality of their work may be the same. Also, some clients do not participate in the WWA system at all.


This is all true, but the same can be said of KudoZ. Not every good translator participates in KudoZ, not every good translator is able to respond first or respond with a sufficiently "helpful" answer, etc. It's a trade-off, and the ProZ.com founder chose that trade-off knowingly. Why can't the same not be done for WWAs?

If WWAs affected directory ranking, you'd still get translators who specifically choose not to collect WWAs, in the same way that some translators choose not to collect KudoZ points, even though it is well-known that KudoZ points affect your ranking, but that should not make WWAs any less reliable for ranking purposes than KudoZ points.

Yes, some good translators do not have those types of clients, but some good translators are not good at answering terminology questions even though they are very good at creating very good translations (and being good at answering terminology questions does not really make you a good translator anyway), so even with KudoZ points, not all translators have an equal chance.


Josephine Cassar
Morano El-Kholy
 
Janet Muehlbacher
Janet Muehlbacher  Identity Verified
Local time: 09:42
German to English
+ ...
why not list the Kudos points separately? Feb 26, 2020

and do without any ranking system. As some have pointed out, many good translators don´t participate in the Kudos answering, maybe they don´t have time for it! As paying members they all deserve exposure to potential clients.

Morano El-Kholy
 
Gabriella Vento
Gabriella Vento  Identity Verified
United States
Local time: 01:42
Member (2015)
English to Hungarian
+ ...
TOPIC STARTER
The true value Feb 26, 2020

Thank you for taking the time to share your opinions.

I cannot imagine any easy ways to earn a WWA. In my case half of my 22 WWAs has come from projects of 5k words or bigger, and one WWA has come from a 206k tender project that I've worked for 4 months on.
Kudos are hard to get too, but I suppose it might have been easier at the beginnings with only a few translators in a language pair.

What is the "true value of a translator" on Proz? We have to remember, this
... See more
Thank you for taking the time to share your opinions.

I cannot imagine any easy ways to earn a WWA. In my case half of my 22 WWAs has come from projects of 5k words or bigger, and one WWA has come from a 206k tender project that I've worked for 4 months on.
Kudos are hard to get too, but I suppose it might have been easier at the beginnings with only a few translators in a language pair.

What is the "true value of a translator" on Proz? We have to remember, this is primarily a business, we are here to make a living by providing the services to our customers. Therefore the primary measure of our value here should be how many of our customers are satisfied enough to Willing to Work with us Again.
Collapse


Morano El-Kholy
 
Laureana Pavon
Laureana Pavon  Identity Verified
Uruguay
Local time: 05:42
Member (2007)
English to Spanish
+ ...

MODERATOR
Disagree Feb 26, 2020

I completely disagree with the WWA ranking.
In addition to the confidentiality issue mentioned above, only translation agencies hiring through ProZ leave feedback on the site.
I work with many clients who have never even heard of ProZ or any other translation portal.


Michele Fauble
Christel Zipfel
Maria Teresa Borges de Almeida
Helen Shiner
 
Katalin Szilárd
Katalin Szilárd  Identity Verified
Hungary
Local time: 09:42
English to Hungarian
+ ...
Kudoz is optional - Receiving WWAs are not optional at all - Terminology is essential Feb 26, 2020

Samuel Murray wrote:

Not every good translator participates in KudoZ, not every good translator is able to respond first or respond with a sufficiently "helpful" answer, etc.

If WWAs affected directory ranking, you'd still get translators who specifically choose not to collect WWAs, in the same way that some translators choose not to collect KudoZ points, even though it is well-known that KudoZ points affect your ranking, but that should not make WWAs any less reliable for ranking purposes than KudoZ points.

... but some good translators are not good at answering terminology questions even though they are very good at creating very good translations (and being good at answering terminology questions does not really make you a good translator anyway)




Participating in Kudoz is optional.

Receiving WWAs are not optional at all.

Terminology is essential in translation. Not just in "technical (specialized)" translations.
Even in marketing or literary fields there are exciting Kudoz questions, where you notice colleagues, whose answers are wittier, more relevant, cool, more creative.
I cannot talk about other language pairs than mine, but in Hungarian Kudoz pro questions are asked mainly. Partly this is because English and Hungarian are so different, word order is different (in more complex sentences it is more difficult to decide which part refers to where), we use different idioms, we have a different culture etc.

I have never met a translator, who was a good translator, but she/he was not good in terminology.
I'm not particularly talking about Kudoz, but when you ask a colleague "Can you help in this term question? Do you have any ideas? How would you solve this language issue? etc."
It is like a puzzle sometimes.
This is our job. This is what makes us linguists.


 
jyuan_us
jyuan_us  Identity Verified
United States
Local time: 04:42
Member (2005)
English to Chinese
+ ...
I think you can get a WWA with a job of any volume Feb 26, 2020

Gabriella Vento wrote:

Thank you for taking the time to share your opinions.

I cannot imagine any easy ways to earn a WWA. In my case half of my 22 WWAs has come from projects of 5k words or bigger, and one WWA has come from a 206k tender project that I've worked for 4 months on.
Kudos are hard to get too, but I suppose it might have been easier at the beginnings with only a few translators in a language pair.

What is the "true value of a translator" on Proz? We have to remember, this is primarily a business, we are here to make a living by providing the services to our customers. Therefore the primary measure of our value here should be how many of our customers are satisfied enough to Willing to Work with us Again.




As long as you deliver a job on time and no quality issue is raised by the client, he will be willing to make a WWA entry, if you request him to do so. It won't be hard for anyone in the business for, say, 10 years, to get 20 - 30 WWAs. Those in this business for a shorter period of time will have a hard time collecting as many WWAs. If WWAs are used for ranking purpose, it will create inequality and unfairness for many of the linguists who haven't been in this business for a long time, and for some linguists with other circumstances that made it hard for them to get WWAs.


Elizabeth Tamblin
Maria Teresa Borges de Almeida
 
Katalin Horváth McClure
Katalin Horváth McClure  Identity Verified
United States
Local time: 04:42
Member (2002)
English to Hungarian
+ ...
@Samuel - this is my last post as this is a limited thread Feb 27, 2020

Translator A services 30 clients a month, B services 3. Both translate the same amount and participate in WWA. A can request 30 WWAs, B can request only 3. The resulting number of WWAs is different and unrelated to client satisfaction.

KudoZ is different, because once you decide to provide an answer, the only thing that is in your way of getting points is others providing better answers. (Yeah, we can get into some fairness issues, etc. but in the grand scheme of things, the points
... See more
Translator A services 30 clients a month, B services 3. Both translate the same amount and participate in WWA. A can request 30 WWAs, B can request only 3. The resulting number of WWAs is different and unrelated to client satisfaction.

KudoZ is different, because once you decide to provide an answer, the only thing that is in your way of getting points is others providing better answers. (Yeah, we can get into some fairness issues, etc. but in the grand scheme of things, the points are related to the quality of the answer.)

I am not a fan of using KudoZ points in the directory ranking the way it is used now. In fact, I have proposed over the years a few different ways/formulas to consider, e.g. incorporating the "reliability ratio" (the percentage of answers accepted), and including factors to "tune down" the value of old points, thereby giving a chance to newer members to "catch up". You may be able to find old forum posts on that topic.

That said, I think using the number of WWAs as the main ranking criteria is a bad idea. In fact, using any single parameter is probably a bad idea. Henry tried to come up with a combined formula for ProZ Find, but it failed spectacularly, because there were arguments against every proposed combination. Perhaps the perfect recipe was not proposed, but I have a feeling, it would be hard to come up with something that everyone feels fair.

Perhaps a totally radical idea, where there is no ranking at all would be the best. In such a system, the client could specify search parameters or filters, but beyond that the order of presenting the list would be random (within paying vs. non-paying groups).
Collapse


 
Katalin Szilárd
Katalin Szilárd  Identity Verified
Hungary
Local time: 09:42
English to Hungarian
+ ...
@ jyuan @Gabriella @Katalin Feb 27, 2020

jyuan_us wrote:

As long as you deliver a job on time and no quality issue is raised by the client, he will be willing to make a WWA entry, if you request him to do so. It won't be hard for anyone in the business for, say, 10 years, to get 20 - 30 WWAs.


This is not true. Especially in medical translation.
I was wondering why I don't receive WWAs, then I asked some of my former clients.

Agency: This was a brilliant job again. Thank you, Katalin.
Me: Would you mind writing this as a WWA?
Agency: No, our agency is prohibiting us to give references for our translators.
Me: Do you accept BB ratings?
Agency: Yes.
Me: Don't you think this is unfair?
Agency: Sorry.
End of the story.

- Since Proz Find is (?was?) not working normally many times, as it is used to,

- Since as far as I'm concerned on the mobile version, translators still cannot be contacted (only if the client registers),

- Since there is almost NO visitors from Proz Find, and almost none from the old directory (I'm nr. 2 in the directory according to the Kudoz (!) and nr. 1 in in my main speciality: medical, so this is very weird). Many other colleagues complained more times as well that they didn't receive any visitors from Proz Find and from the other directory meanwhile their rankings (due to Kudoz) were very high,

- Since I have seen that colleagues who were on the first page in the top part due to their Kudoz points, decided to let their subscriptions expire, because they did not receive any visitors (!!) from the directories and did not receive any client emails from the directories (I'm not talking about mass emails, but emails addressed to that specific translator)...

In the past few years (4-5) many of us ranked as nr. 1-20 (not just in my language pair) according to our Kudoz points receiving almost no visitors, or clients contacting us from the directories.

On the other hand maybe those who have less Kudoz points may think we receive the most visitors and jobs due to our Kudoz points...

Meanwhile proz clearly stated that they put the "Post a job" icon in the directory to navigate clients to the job board, because job board generates more activity, and activity means more money for proz.


So if things don't change positively in this, sooner or later translators will check for a different way to find clients. I'm not talking about the job board...
Best clients came from the directories....in the old days....
and I'm not talking about mass emails.

Katalin Horváth McClure wrote:
Perhaps a totally radical idea, where there is no ranking at all would be the best.


I don't think it would be good and fair, if I think about how many times I had to write support tickets due to "bugs" in directories. Also if it works like the "featured translator" option in the home page, probably some translators will be always listed as last or missed out...
Also clients want to find the best translators, not just randomly someone, who is listed there.



[Edited at 2020-02-27 08:36 GMT]

[Edited at 2020-02-27 09:06 GMT]

[Edited at 2020-02-27 09:20 GMT]


 
Sheila Wilson
Sheila Wilson  Identity Verified
Spain
Local time: 08:42
Member (2007)
English
+ ...
My problems with WWA Feb 27, 2020

1. I think WWA might have changed a while back, but in the past all clients could do was tick a box to say they were satisfied, and they could optionally say something nice. No negative entries were allowed. So, how could we ask for feedback when their hands were so comprehensively tied? I don't feel it's fair to ask. So the few WWA I have are from clients who know the ProZ.com system or asked where they could leave feedback. There must be a lot of us who feel the same way. Integrity shouldn't b... See more
1. I think WWA might have changed a while back, but in the past all clients could do was tick a box to say they were satisfied, and they could optionally say something nice. No negative entries were allowed. So, how could we ask for feedback when their hands were so comprehensively tied? I don't feel it's fair to ask. So the few WWA I have are from clients who know the ProZ.com system or asked where they could leave feedback. There must be a lot of us who feel the same way. Integrity shouldn't be a disadvantage.

2. Many of us have mainly long-term clients rather than working with dozens of new agencies for a short time. Long-term clients are surely a clear indication of quality, and shouldn't be a disadvantage.

3. Many less established freelancers are quite happy to do trade-offs with the not-so-reputable agencies who say "You give us a good LWA ranking, and we'll pay you" (even though it's against the rules here). Even the more respectable agencies often say "You give us a good LWA ranking and we'll give you an equally good WWA". They don't mean it as a bribe or anything, but I won't go along with it. Again, integrity shouldn't be a disadvantage.

Let's face it, no way of ranking is going to be perfect. But with computers being so good at data gathering and manipulation, I think it would be best for ProZ.com to use all that data they hold on us to come up with aggregate ranking scores, taking all sorts of things into account.
Collapse


Elizabeth Tamblin
Maria Teresa Borges de Almeida
 
Elizabeth Tamblin
Elizabeth Tamblin  Identity Verified
United Kingdom
Local time: 08:42
French to English
Disagree Feb 27, 2020

The system you propose would not work for me. I work with a limited number of clients and would not want to ask them for a WWA review. I have spent quite a bit of time building up KudoZ points, and one of the main reasons why I continue to pay for ProZ membership is that I do not want to lose my ranking in the directory.

Josephine Cassar
Michele Fauble
Maria Teresa Borges de Almeida
 
Amin Zanganeh Inaloo
Amin Zanganeh Inaloo  Identity Verified
Belgium
Local time: 09:42
Member (2015)
English to Persian (Farsi)
+ ...
Disagree May 15, 2020

Hi,

Maybe this is the case for your language pair. But, lots of unfair attempts are made in regard Kudoz terms and points (including peer reviewing by colleagues or friends and also unreal profiles posting Kudoz questions and selecting their intended answerers in an unfair way) in my language pair. Moderators can also do limited things in this regard in cases we report such actions. Kudoz-based directory ranking is no way a ideal and fair.

Vanda Nissen wrote:

Gabriela,

I see your point, and it might be a good idea to make WWA more visible, however I do not agree that they should prevail over Kudoz points. Technically, you can get 30 WWA entries in one month, but it is impossible to get a lot of Kudoz points straight away. Answering questions takes time and effort. In fact, it tells more about a translator than mere WWAs. This is an unbiased opinion because I have both.

Although Kudoz debate sometimes get heated, I have not witnessed a single translator in my pairs voting for a friend. If you have witnessed such an unethical behaviour, you should certainly report it to your moderator.


 
Pages in topic:   [1 2] >


To report site rules violations or get help, contact a site moderator:


You can also contact site staff by submitting a support request »

Default ranking in Translators Directory






Trados Business Manager Lite
Create customer quotes and invoices from within Trados Studio

Trados Business Manager Lite helps to simplify and speed up some of the daily tasks, such as invoicing and reporting, associated with running your freelance translation business.

More info »
CafeTran Espresso
You've never met a CAT tool this clever!

Translate faster & easier, using a sophisticated CAT tool built by a translator / developer. Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools. Download and start using CafeTran Espresso -- for free

Buy now! »