Statistics Not Showing After Upgrade To 2.8.8 Options

[michel_b]

Hi

We've had our install of openx 2.8.4 hacked, so we upgraded to 2.8.8. During upgrade there were some plugin errors, which I fixed by copying the plugin folders accross. Ads are now serving out fine, but statistics aren't updating ever since the update on friday.

I've had a good rumage through all the various posts regarding this, and tried

- checking DB tables for locks/errors - nothing
- debug.log for errors - nothing
- apache error log for errors - nothing

Don't get any errors on login, or anywhere else, just no statistics.

I've traced the data through the migration flow, and it would appear the problem lies with the maintenance script, or rather with the migrateBucketData part of it.

AFAIK, the data flow is

ox_bkt_m contains the raw data, which is up to date to, and still being updated
ox_intermediate_ad contains intermediate data, this was last updated on the 24th, at the time of migration
ox_data_ad_hourly contains the statistics data, which is what the statistics pages use. This was also last updated on the 24th.

Hence the issue seems to lie with migrating the data from the raw table to the intermediate and stats table.

I ran the migrateBucketData.php script manually from the php-cli, and it just completes, no errors.

switched debug level to max, and this is all I got

CODE

Mar 01 12:50:43 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Running Maintenance Engine
Mar 01 12:50:43 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Running Maintenance Statistics Engine
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task begin: OX_Maintenance_Statistics_Task_SetUpdateRequirements
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Maintenance start run time is 2012-03-01 12:50:43 UTC
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Getting the details of when maintenance statistics last ran on the basis of the operation interval
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Maintenance statistics last updated intermediate table statistics to 2012-03-01 12:44:59 UTC
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Current time must be after 2012-03-01 12:49:59 UTC for the next intermediate table update to happen
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Getting the details of when maintenance statistics last ran on the basis of the hour
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Maintenance statistics last updated final table statistics to 2012-03-01 11:59:59 UTC
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Current time must be after 2012-03-01 12:59:59 UTC for the next intermediate table update to happen
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - At least 60 minutes have NOT passed since the last final table update
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  - Maintenance statistics will be run
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  - The intermediate table statistics will be updated
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task complete: OX_Maintenance_Statistics_Task_SetUpdateRequirements
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task begin: OX_Maintenance_Statistics_Task_MigrateBucketData
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  There are no installed plugins that require data migration
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task complete: OX_Maintenance_Statistics_Task_MigrateBucketData
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task begin: OX_Maintenance_Statistics_Task_DeDuplicateConversions
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Deduplicating conversions between 2012-03-01 12:45:00 UTC and 2012-03-01 12:49:59 UTC
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Rejecting conversions with empty required variables between 2012-03-01 12:45:00 UTC and 2012-03-01 12:49:59 UTC
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task complete: OX_Maintenance_Statistics_Task_DeDuplicateConversions
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task begin: OX_Maintenance_Statistics_Task_ManageConversions
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Selecting conversion data for migration to the "old style" intermediate table for
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]    conversion in the range 2012-03-01 12:45:00 UTC to 2012-03-01 12:49:59 UTC
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task complete: OX_Maintenance_Statistics_Task_ManageConversions
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task begin: OX_Maintenance_Statistics_Task_SummariseFinal
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Saving request, impression, click and conversion data into the final tables.
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task complete: OX_Maintenance_Statistics_Task_SummariseFinal
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task begin: OX_Maintenance_Statistics_Task_LogCompletion
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Logging the completion of the maintenance statistics run
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  Logging the maintenance statistics run report
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task complete: OX_Maintenance_Statistics_Task_LogCompletion
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task begin: OX_Maintenance_Statistics_Task_ManageCampaigns
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  - Managing (activating/deactivating) campaigns
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Requesting campaigns to test for activation/deactivation
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]  - Found 68 campaigns to test for activation/deactivation
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]    - Selecting impressions, clicks and conversions for this running campaign ID = 2677
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]    - Sending campaign 'soon deactivated' email
Mar 01 12:50:48 +0000 OX-maintenance-4f4f70a3a3ebb [    debug]    - Sending campaign 'soon deactivated' email
Mar 01 12:50:48 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Task complete: OX_Maintenance_Statistics_Task_ManageCampaigns
Mar 01 12:50:48 +0000 OX-maintenance-4f4f70a3a3ebb [     info]  Maintenance Statistics Engine Completed (Started at 2012-03-01 12:50:43 UTC)

The line
Mar 01 12:50:44 +0000 OX-maintenance-4f4f70a3a3ebb [ debug] There are no installed plugins that require data migration

prompted me to check the plugins, none of which were enabled. I enabled
openXVideoAds1.2
openXWorkflow0.0.1-RC2
openXDeliveryLog1.0.9
openXInvocationTags1.0.8
openXMaxMindGeoIP1.0.4
openXDeliveryCacheStore1.0.6
openXReports1.0.7
openXDeliveryLimitations1.0.13
openXBannerTypes1.0.1

Now the [ debug] There are no installed plugins that require data migration message is gone, but still no stats..

I've spend 3 days on this now and bar really taking apart the openx code I'm rather stuck, hence my plea for help.

Any ideas what might be causing this, or how I go about finding out even?

Thanks for your help

Michel

[michel_b]

Hi

Made some headway, did some checking on the plugings and noticed on the plugin report in

/www/admin/maintenance-plugins.php

There were file not found errors.

Checked the original code for further plugin related folders, found

/etc/plugins
/plugins
/plugins/etc
/var/plugins/DataObjects

Copied all the above, and set the permission to 777, and now it started log again!

Only seems to log new stats however, not the stats since the 24th. tried the migrateBucketData script again, and rollUpStats.php, and now the stats have totally dissapeared, even the historic ones.

They're still in data_bkt_m and the intermediate table, but ox_data_ad_hourly is now empty..

Anyone know how I repopulate the ox_data_ad_hourly table with one of the tools?

Thanks

Michel

Also I set the permissions on plugins to 777.