Share

SourceForge.net

Tracker: Support Requests

9 Master bug report: download statistics innacuracies - ID: 745577
Last Update: Comment added ( burley )

About this bug report: This support request is a
'master bug report', submitted by SourceForge.net staff
to mark a known issue which affects a number of
SourceForge.net users. This single support request
will be used to replace all other existing bug reports
and support requests which have been submitted about
this particular issue.

Attention SF.net staff: When closing this support
request or posting issue updates, please keep in mind
that this request is being monitored by SourceForge.net
users, and is listed on the Site Status page (Known
Bugs section). Please leave this support request open
until the referenced issue has been completely addressed.

Issue at hand: Download statistics for projects are
inaccurate (either 0 or seemingly low) for a large
number of projects. This issue appears to have started
on or around May 19, 2003 and to be continuing through
at least May 27, 2003 and most likely will continue
until this ticket is closed.

In the event that you should have questions or concerns
regarding the status of this issue, please add a
comment to this support request.

Initial Support Requests consolidated under this Master
Bug Report:
745543, 745514, 745477, 745466, 745470, 745215, 745410,
745355, 745315, 744711, 744977, 745057, 744785, 744737,
743983, 743949, 743995, 744161, 744248, 744312, 744390,
744059, 744159, 741547, 741519, 741719, 742161, 742331


SourceForge.net Site Docs ( sitedocs_mngr ) - 2003-05-29 15:41

9

Closed

None

David Burley

Project and Site Statistics

Second Level Support

Public


Comments ( 40 )

Date: 2006-03-14 21:42
Sender: burleySourceForge.net SubscriberSourceForge.net Site Admin

Logged In: YES
user_id=597273

Greetings,

Due to the age of this issue, the impact at this time, and
the possibility for resolution, this issue is being closed.

Thank you,

David Burley
Quality of Service Analyst, SourceForge.net


Date: 2003-08-04 18:11
Sender: rhavyn

Logged In: YES
user_id=31362

Hello,

As many people have probably noticed, download statistics
are being recomputed as I type this. To avoid confusion,
I'd like to alert people to a small deficiency in the
process of recalculating everything.

Due to a small technical issue, the new stats system is only
running on days which have already been calculated by the
old stats system. This is not caused by any issues in the
new stats system and once stats have been correctly
recalculated the new stats system will be doing the nightly
processing. However, that does mean that while stats are
being recalculated, there will be a short period of time
during which the incorrect download counts will be visible
from your project stats page. This is a temporary issue and
will no long happen once stats have been correctly recalculated.

Thanks,
Chris Conrad
SourceForge.net Engineer



Date: 2003-07-14 18:34
Sender: moormanSourceForge.net SubscriberSourceForge.net Site Admin

Logged In: YES
user_id=152443

2003-07-14 Status Update:

Download statistics are now being recomputed; pending
completion of the download statistics corrections, aggregate
statistics will also be recomputed. No ETA is being
established as to when this work will be completed; results
of this work will be reviewed for quality assurance purposes
as the rebuilding of download stats progresses.


Date: 2003-06-26 14:28
Sender: moormanSourceForge.net SubscriberSourceForge.net Site Admin

Logged In: YES
user_id=152443

2003-06-26 Status Update:

The SourceForge.net engineer assigned to this task has been
working to review the validity of log data; all captured
data appears to be valid.

The underlying issue with the code base has been identified
(this has been verified to be a log parsing problem) and is
being resolved.

Testing of the updated code base and reprocessing of stats
data (which has been validated) is expected to begin early
next week.

This bug report and the site status page will be updated
once the process of repopulating download statistics and
aggregated statistics (activity ranking) begins on the live
database (expected to begin after completion of the needed
code fixes and a final test run of the new code); or no
later than two weeks from today.

Your patience is appreciated.

Jacob Moorman
Quality of Service Manager, SourceForge.net


Date: 2003-06-19 18:25
Sender: burleySourceForge.net SubscriberSourceForge.net Site Admin

Logged In: YES
user_id=597273

All SourceForge.net Master Bug Reports (including this one)
have now
been locked down as to prohibit further comments from
SourceForge.net
end-users. From this point forward, only updates by
SourceForge.net
staff should be posted to this request.

This change has been made by popular request; each comment
added to this
request will generate an email to all users monitoring this
request
(these comments, on some master bug reports, have generated a
significant amount of email not of direct benefit to the
recipients).
Most of the people who monitor master bug reports do so as
that they may
determine the status of issues that impact their project;
thus, these
master bug reports will now only accept updates by
SourceForge.net
staff.

Unless a specific request has been made for additional
information from
your project, it is safe to assume that the SourceForge.net
team is
aware of the details of this issue, which projects it
impacts and how
those projects are impacted. You need not contact us with
those details
(we have consolidated this issue on to a single ticket
/because/ we have
those details already and do not need individual issue reports).

Updates to all outstanding master bug reports will be made
in the next
few weeks; all issues will benefit from regular updates from
that point
forward. The SourceForge.net team does its best to provide
current,
accurate status information, as well as details of available
workarounds
that may be used to minimize the impact of the documented issue.

Should you have questions or concerns regarding this change,
the status
of or details of this master bug report, please contact
Jacob Moorman
(Quality of Service Manager, SourceForge.net) at
moorman@sourceforge.net

Thank you,

SourceForge.net support


Date: 2003-06-18 16:11
Sender: moormanSourceForge.net SubscriberSourceForge.net Site Admin

Logged In: YES
user_id=152443

To further clarify the status of this request:

SourceForge.net staff are aware of the scope of this issue.
We are able to derive which projects are effected and which
are not. We do not need additional information about
whether your project (or other projects on the site) are
afflicted by this bug, nor information regarding the levels
of downloads expected/received/performed by your project.

This request has been locked down as to prevent further
comments by site end-users. We have previously stated
(2003-06-11) that no further information should be posted to
this request (but our statement may have been unclear, so we
are making this statement and locking down the ticket to
prevent further comments). Each comment posted to this
ticket generates an email to every person monitoring this
ticket, including the full text of every previous comment to
this ticket. Thus, "me too" posts and reports of individual
project issues to this ticket are not seen as desirable --
they add to the amount of traffic sent to a large group of
people who are monitoring this request and do nothing to
provide useful information to those parties (the more than
50 people who are monitoring this ticket to determine the
status of the download statistics for /their/ projects). As
stated again, we do have all of the information we need to
resolve this issue and appreciate the assistance provided
previously to ensure that this information was collected in
a complete manner.

Diagnosis, analysis and repair of the download statistics
code is in-progress. As stated previously, the next update
to this request will come no later than 2003-06-26.

Should you have concerns or questions regarding this issue,
please contact me directly at moorman@sourceforge.net

Thank you,

Jacob Moorman
Quality of Service Manager, SourceForge.net


Date: 2003-06-17 14:43
Sender: chirpy

Logged In: YES
user_id=747169

Sadly, we have no download statistics for our project either
from inception to date. The project is: poplimitd. It would be
nice to know how many have downloaded from the site and
sincerely hope the statistics are backdated.

Regards,
Jonathan


Date: 2003-06-16 18:11
Sender: we7u

Logged In: YES
user_id=448632

Download stats for the "xastir" project (45562) are also
thoroughly messed up, starting on May 19th and continuing
through the present. I sure would like to see accurate
numbers for the new release we did last friday!


Date: 2003-06-15 18:52
Sender: mars-man

Logged In: YES
user_id=670243

Hello and have a good time

Downlod and CVS statistics of ncodegen project are zero from
19 May to 3 June and **after that**

Regards
Mars-Man



Date: 2003-06-13 13:15
Sender: sven_weihAccepting Donations

Logged In: YES
user_id=466842

Greetings,

thank you very much for solving this problem for us. My
project is also affected. Even if it is a compareble low
number of projects showing wrong statistic data, this is
still a bit frustrating.

Do you think, it is possible, to display a message on the
statistics page, which says that statics are not correct for
some projects at the moment. This would bring some more
light into project statistics.

For all who are at 0%: Our project is now around 70-80% of
activity without downloads. But we are using SF-CVS which
gets us those 70-80%.

Sven.


Date: 2003-06-12 19:08
Sender: moormanSourceForge.net SubscriberSourceForge.net Site Admin

Logged In: YES
user_id=152443

Greetings,

As promised, this is the status update regarding the
download statistics issue covered by this master bug report.

There are actually two issues covered by this request. The
first issue relates to a statistics collection problem which
took place from the time period of 2003-05-19 through
2003-06-02. Due to this statistics collection problem,
there was a significant absence of download statistics data
for most projects. This issue has been resolved (by rhavyn)
and statistics data for the days in question has been
repopulated. Though most projects now have download stats
for these days, a number of projects are also effected by
the second issue.

The second issue relates to a statistics data extraction
problem (i.e. taking the collected statistics data and
merging it in to our statistics database). This problem is
the source of the most recent comments on this request, but
is actually a separate problem, not technically related to
the initial issues consolidated on this request. This issue
is currently outstanding; projects may encounter a 0
download count or have lower-than-expected download counts.

The remaining issue is code-related and deals with accuracy
of the collected download statistics data. All statistics
data for the affected days and projects has been collected
properly, we believe. Analysis of the issue is pending (a
SF.net engineer is assigned to this task). Pending fixes to
the code, data for one date will be reprocessed and the
results will be manually reviewed by SF.net staff for
accuracy. After this review occurs, we can proceed to
re-run all of the download statistics (and related
aggregations) for the affected dates.

It should be recognized that this is a significant amount of
work and, while the SourceForge.net team has dedicated an
engineer to this project, it will likely persist for several
more weeks (the process of manually re-running stats jobs is
a background-type task that does not require manual
babysitting, but still requires almost a day to perform the
extraction, generation and analysis of a single day of stats
data).

We will try to post more regular status updates to this
ticket. Failing a regular update any sooner, an update will
be posted on 2003-06-26 (two weeks from today). Your
patience is appreciated. If you have specific concerns
which you believe have not been addressed here, please
contact me directly via email at moorman@sourceforge.net

Thank you,

Jacob Moorman
Quality of Service Manager, SourceForge.net


Date: 2003-06-11 16:38
Sender: moormanSourceForge.net SubscriberSourceForge.net Site Admin

Logged In: YES
user_id=152443

Greetings,

I will touch base with the SF.net engineering team today to
check on the status of this issue. Please sit tight -- it
is not necessary (nor desirable, at this point, since we
have enough information to proceed) to have additional posts
regarding download statistics anomalies. Updates will
follow in the next 24-48 hours (1-2 business days) including
current issue status and ETA for resolution. Your feedback
to date and patience have been appreciated.

Jacob Moorman
Quality of Service Manager, SourceForge.net


Date: 2003-06-11 15:14
Sender: jgrahamc

Logged In: YES
user_id=578491

I'm going to an my "echo" to the list. I've just shipped the
latest version of my project POPFile and I know I'm going to
get many, many downloads but the lack of download
information is going to be very disappointing...

Please let us know the status. Silence is worse than
hearing "it will take us a month".

John.



Date: 2003-06-11 15:05
Sender: dglasserSourceForge.net Donor

Logged In: YES
user_id=547875

I want to echo the sentiments of Bruce Korb and Giulio
Toffoli. Please at least give us some idea of what is going on
with this bug. Do you expect to have it fixed soon, or are
there more important problems right now that are preventing
this from even being worked on? Either answer is fine, I'd just
like to know. I've been holding back on a release waiting for
this to get fixed, because, as Giulio mentioned, knowing that
people are downloading our software is about the only reward
we get for writing it, aside from being able to use it ourselves.

If this isn't expected to be fixed for a long time, please let us
know and I'll go ahead and release my latest version. Don't
interpret this as a complaint about the service--I think
sourceforge is an awesome service, and I don't want to sound
like I'm whining about something I'm getting for free.



Date: 2003-06-11 14:34
Sender: gt78

Logged In: YES
user_id=616744


Dear Davis, as I have already written to you...

My project, as many projects hosted on SF, is affected
by a very boring problem with download statistic. My
program is not downloaded from 19 May? Obviously not,
0 is only the statistic number. But it's only a number?


This is not a problem that regards only statistics as
number.
For many project administrators and developers the
statistics are not simply numbers, but they are a form of
gain for their work.
If download statistics don't work, many projects don't
release new versions, because they don't know the
entity of success of their work.


IMHO the download statistic issue is a not a simply
problem regarding a mere calculation, but it's a disease
to vanquish in order to garantee new releases.


Giulio Toffoli
(http://ireport.sf.net)



Date: 2003-06-11 13:59
Sender: bkorbAccepting Donations

Logged In: YES
user_id=19940

Dear Source Forge,

It is time for an update, even if to say "we don't under-
stand the problem" or "we don't have time to fix it".
Many people are contributing to this report to "help prove"
that stats are still broken. It is pretty clear enough.
Your last report was a week ago when it was claimed to be
fixed and your status page shows everything being fine, tho
there is a two week old notation hinting "maybe not for
some." Prognosis, please?


Date: 2003-06-11 07:52
Sender: dnovatchev

Logged In: YES
user_id=547596

The FXSL project -- //sourceforge.net/projects/fxsl

is still affected -- there are 0 downloads reported for any
file since 19 May.

Today is 11 June -- 8 days after 3 June, so the claims that
this bug has been fixed and statistics will be correct
starting from 3 June -- these claims are not true.

Please, fix this bug, or add third party counters to the
download pages.

Dimitre Novatchev.


Date: 2003-06-10 20:55
Sender: dandubinsky

Logged In: YES
user_id=552177

My project still has a problem as
well. /sourceforge.net/projects/salmon. Please fix this bug,
it's been out there for a month. I know source forge considers
stats bugs low priority, but we find them extremely useful as
a gauge of how well things are going and the timing of new
releases. After this long, the priority should go up I think.


Date: 2003-06-10 20:37
Sender: dutchrollSourceForge.net DonorAccepting Donations

Logged In: YES
user_id=545547

(Ref. 743949) Well, contrary to what I thought a couple of
days ago, my download statistics still seem to be unreliable.
Since june 3, my download numbers are not zero anymore,
but exactly what they represent is a true mystery. I have
tried to verify the statistics by downloading some of my files
myself (even using different mirrors, just to be sure), but
those downloads were _not_ added to the counters.

Regards,
Marc Rauw

(PS: in response to dglasser's post: a combination of large
download numbers and low page view numbers will occur
when a project links to these files directly from a website that
is hosted somewhere else, or that does not contain the
SourceForge banner. Tampering with these statistics is
possible, but faking 1500 downloads or more would be
downright ludicrous. ;)


Date: 2003-06-09 19:17
Sender: toddrun

Logged In: YES
user_id=588095

My apologies for posting 751507. It should be added to this
issue. Visual Swing isn't recording downloads (and I
suspect, though I can't prove that Route Ruler is having the
same issue).

Sorry again for posting a known issue.


Date: 2003-06-09 15:36
Sender: mightye

Logged In: YES
user_id=735268

Still seems to be happening to my project on June 9th.
http://sourceforge.net/projects/lotgd/


Date: 2003-06-09 02:11
Sender: dglasserSourceForge.net Donor

Logged In: YES
user_id=547875

earnie's post about DEV-C++ prompted me to check 4 or 5 of
the currently listed top-10 downloaded programs, and every
one I checked showed a HUGE jump in their download volume
on May 19th which has held steady ever since. That has to
be related to all of the other projects that dropped suddenly
to zero per day and stayed there (like my project, qform.)

While we're on the subject, does SF have checks in place
that prevent people from gaming their download stats? I
noticed one project, that even before May 19th was getting
around 1500 downloads a day but only 40 or 50 page views.
This isn't an accusation, but that seems a little suspicious.



Date: 2003-06-08 19:53
Sender: earnie

Logged In: YES
user_id=15438

Amazing DEV-C++ has 527,805 downloads is it possible that
the method isn't properly determining the project for the
download?



Date: 2003-06-08 17:36
Sender: jgrahamc

Logged In: YES
user_id=578491

Looks still broken to me. POPFile usually gets 100-300
downloads per day, it's be saying 0-5 since May 19 and
nothing has changed in the last couple of days.

John.



Date: 2003-06-07 12:52
Sender: zirkon13SourceForge.net Subscriber and Donor

Logged In: YES
user_id=575469

Hello Chris Conrad!

I'm sorry but I have to disillusion you about
having fixed the download-counter statistic bug
on sf.net.

As you can clearly see by the project yappa-ng
with 0 downloads again the bug is there as
before. Since the day you wrote the bug to be
fixed I made personally 3 downloads EACH DAY just
to check it - and as you can clearly see there
are only zeros!

btw: the CVS-Bug is still there too - I have
multiple commits a day, bacause I'm debugging a
lot and there are 0 cvs-activities too. But thats
another bugreport!

So please keep on debugging the download counter
statistic bug and do not close this Master
Bugreport!

best regards

Fritz Berger
Project Admin yappa-ng



Date: 2003-06-06 07:22
Sender: tevfikSourceForge.net Subscriber

Logged In: YES
user_id=562369

Hi,

I am not so sure about that the problem is gone. I released a
new version of logrep on June 2nd. Page views seems to be
correct. However, download statistics is a sad chapter !

This problem exists since mid-May. Please do something!

Best regards


Date: 2003-06-05 22:49
Sender: nobody

Logged In: NO

As for us (the perlqt project), it's still totally broken...
All downloads stop abruptly on May 19th.
That's so depressive 8(


Date: 2003-06-05 22:19
Sender: earnie

Logged In: YES
user_id=15438

The counts for the MinGW project from May 24th through today
are still not believable. Seem low compared to prior to May
24th.

Earnie


Date: 2003-06-05 20:19
Sender: bkorbAccepting Donations

Logged In: YES
user_id=19940

I don't think so. I just barely released a new autogen and
that always generates several day's worth of multiple-10's
of downloads. I also verified the operations of some of
the download mirrors on Tuesday. The stats say that all
that activity adds up to one download the whole day.
I don't think so. Don't forget the omitted CVS activity,
either, please.


Date: 2003-06-05 20:01
Sender: rhavyn

Logged In: YES
user_id=31362

Hello,

The bug which was causing download statistics to be
miscounted has been fixed. Stats from June 3rd onwards
should be correct. We are currently in the process of
correcting the download statistics for the days of May 19th
- June 2nd. This process should be completed by start of
business tomorrow. Thank you for your patience while we
deal with this problem.

Chris Conrad
SourceForge.net Engineer



Date: 2003-06-05 18:55
Sender: dutchrollSourceForge.net DonorAccepting Donations

Logged In: YES
user_id=545547

(Ref. 743949)

It seems that download counting for my Dutchroll project has
resumed as of june 3. The activity percentage has also risen
back from the depressing 0% of the last couple of weeks, so
I hope the problem has been fixed.

Regards,
Marc Rauw


Date: 2003-06-05 15:32
Sender: mgrouchAccepting Donations

Logged In: YES
user_id=643949

I've noticed more strange things with statistics (apart from
that it doesn't count all downloads).

Suddenly download counters on some of my files went down
to zero.

Downloads count on summary page and 'files' page do not
match either.

It would be also nice if downloads counter incremented
immediately after download (and not few days later).




Date: 2003-06-03 14:14
Sender: bkorbAccepting Donations

Logged In: YES
user_id=19940

Don't forget CVS stats while you are at it.
Missing for AutoGen on 5/31/03. And, please,
don't rush the stats job. Please fix it so
it is reliable. Thanks! - Bruce


Date: 2003-06-02 21:30
Sender: alanhAccepting Donations

Logged In: YES
user_id=2372

If it's been outstanding since May 19th.....

When is it going to get fixed ?


Date: 2003-05-31 18:13
Sender: bronger

Logged In: YES
user_id=442234

I've always found download statistics rather shaky. In my
project (tbookdtd), they suddenly exploded (as with some
other projects at that time), and my tiny project was even
on the top ten list. Moreover, we've always had single days
when most projects (even the very big ones) had zero downloads.

The thing that download counting doesn't start immediately
after uploading is also unfortunate behaviour in my opinion.


Date: 2003-05-31 17:38
Sender: gt78

Logged In: YES
user_id=616744

it is possible to know which is exactly the problem of
statistics? They have always worked correctly....

TIA
Giulio


Date: 2003-05-31 04:19
Sender: johannesprixAccepting Donations

Logged In: YES
user_id=511578

Our project is affected too, but we haven't yet files our
own support request. Should do that?
But since it might be a site-wide problem, I guess it will
go away sooner or later anyway :)


Date: 2003-05-30 11:06
Sender: gt78

Logged In: YES
user_id=616744

Request bug 745916 should be added :-)

Giulio


Date: 2003-05-30 09:26
Sender: hirogen2

Logged In: YES
user_id=522048

Addition to the above: Yeah, if they are seemingly low, there
is *one* day (each week or whatever), where the download
count is very high then. This may not be true for all situations.


Date: 2003-05-30 02:33
Sender: mgrouchAccepting Donations

Logged In: YES
user_id=643949

I've noticed that too for my XmlStarlet Project.
I hope you will fix it as it is depressing to see project
downloads numbers going down after every new release.
:)



Attached File

No Files Currently Attached

Changes ( 6 )

Field Old Value Date By
status_id Open 2006-03-14 21:42 burley
assigned_to enhancement 2006-03-14 21:42 burley
close_date - 2006-03-14 21:42 burley
escalated artf17788 2006-01-09 15:20 enhancement
assigned_to jaybonci 2006-01-09 15:15 moorman
assigned_to rhavyn 2005-05-05 15:56 moorman