API call returning an error: Couldn't find requested table


#1

Hey guys I am getting this error when calling CustomVariables.getCustomVariables over API:

The requested table (id = 7) couldn’t be found in the DataTable Manager

Could you point me in the right direction to troubleshoot this error? What could cause this error?

Thanks, Fedor


#2

I tracked the problem down to date range.

Calling “date=2011-10-10,2011-11-24&period=range” returns error above

However, calling “date=2011-10-24,2011-11-24&period=range” works fine

Any ideas? Could it be because piwik was installed after 2011-10-10 and there is no data for that date?

Thanks, Fedor


(Matthieu Aubry) #3

Are you using piwik 1.6?


#4

Yes 1.6


(Matthieu Aubry) #5

Does the error only show when you include older days? Or does it also sometimes show when including normal days?


#6

Ha you right! It does come up here and there even for existing dates. Is it connected to archiving in any way? So when data requested is already archived, it will return the error? Just a shot in the dark, can’t think of why it would return error sporadically.

Thanks, Fedor


(Matthieu Aubry) #7

Could you please send me the full dump of your mysql database, and also the list of URL requests that trigger the errors?

This is a difficult bug to debug. Thanks


#8

I am facing the same issue with current version 1.7.1. I get intermittently The requested table (id = 4) couldn’t be found in the DataTable Manager

Please advise.


(Matthieu Aubry) #9

When exactly does it happen? does it happen when you click on a row and it loads the subtable and then display the error?

sgoteti, please apply the following patch:


Index: core/DataTable/Manager.php
===================================================================
--- core/DataTable/Manager.php	(revision 5976)
+++ core/DataTable/Manager.php	(working copy)
@@ -73,6 +73,7 @@
 	{
 		if(!isset($this->tables[$idTable]))
 		{
+			debug_print_backtrace();
 			throw new Exception(sprintf("The requested table (id = %d) couldn't be found in the DataTable Manager", $idTable));
 		}
 		return $this->tables[$idTable];

What is the output then?

It is the first time in a very long time that this bug is reported. We would like to get to the bottom of it and fix it.


#10

I am using the REST api. I refresh the call and it happens once in every say 10 times. Seems very random.

756: unexpected token at '#0 Piwik_DataTable_Manager->getTable(4) called at [/var/www/piwik/core/DataTable/Row.php:233]
#1 Piwik_DataTable_Row->sumSubtable(Piwik_DataTable Object ([] => Array ([0] => Piwik_DataTable_Row Object ([c] => Array ([0] => Array ([label] => 5,[2] => 13,[3] => 13,[4] => 1,[5] => 0,[6] => 13,[7] => 0,[11] => 12),[1] => Array (),[3] => ))),[] => ,[] => 1,[] => 0,[] => 1,[] => ,[] => ,[] => Array (),[] => 0,[] => ,[] => ,[] => )) called at [/var/www/piwik/core/DataTable.php:427]
#2 Piwik_DataTable->addDataTable(Piwik_DataTable Object ([] => Array ([0] => Piwik_DataTable_Row Object ([c] => Array ([0] => Array ([label] => CID,[2] => 13,[3] => 13,[4] => 1,[5] => 0,[6] => 13,[7] => 0,[11] => 12),[1] => Array (),[3] => 1)),[1] => Piwik_DataTable_Row Object ([c] => Array ([0] => Array ([label] => FID,[2] => 1,[3] => 1,[4] => 1,[5] => 0,[6] => 1,[7] => 0,[11] => 1),[1] => Array (),[3] => 2)),[2] => Piwik_DataTable_Row Object ([c] => Array ([0] => Array ([label] => MID,[2] => 3,[3] => 3,[4] => 1,[5] => 0,[6] => 3,[7] => 0,[11] => 3),[1] => Array (),[3] => 3))),[] => ,[] => 0,[] => 0,[] => 1,[] => ,[] => ,[] => Array (),[] => 0,[] => ,[] => ,[] => )) called at [/var/www/piwik/core/ArchiveProcessing/Period.php:203]
#3 Piwik_ArchiveProcessing_Period->getRecordDataTableSum(CustomVariables_valueByName, ) called at [/var/www/piwik/core/ArchiveProcessing/Period.php:172]
#4 Piwik_ArchiveProcessing_Period->archiveDataTable(CustomVariables_valueByName, , 1000, 50) called at [/var/www/piwik/plugins/CustomVariables/CustomVariables.php:312]
#5 Piwik_CustomVariables->archivePeriod(Piwik_Event_Notification Object ([_notificationName] => ArchiveProcessing_Period.compute,[_notificationObject] => Piwik_ArchiveProcessing_Period Object ([] => 2159,[] => 5,[] => ,[] => ,[] => ,[] => Piwik_TablePartitioning_Monthly Object ([] => archive_numeric,[] => piwik_archive_numeric_2012_02,[] => 1330387200),[] => Piwik_TablePartitioning_Monthly Object ([] => archive_blob,[] => piwik_archive_blob_2012_02,[] => 1330387200),[] => 1331068876,[] => 1,[] => 1,[idsite] => 1,[period] => Piwik_Period_Range Object ([] => range,[] => Piwik_Date Object ([] => 1330992000,[] => UTC),[] => Array ([0] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1330387200,[] => UTC)),[1] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1330473600,[] => UTC)),[2] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1330560000,[] => UTC)),[3] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1330646400,[] => UTC)),[4] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1330732800,[] => UTC)),[5] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1330819200,[] => UTC)),[6] => Piwik_Period_Week Object ([] => week,[] => Array ([0] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1330905600,[] => UTC)),[1] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1330992000,[] => UTC)),[2] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1331078400,[] => UTC)),[3] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1331164800,[] => UTC)),[4] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1331251200,[] => UTC)),[5] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1331337600,[] => UTC)),[6] => Piwik_Period_Day Object ([] => day,[] => Array (),[] => 1,[] => Piwik_Date Object ([] => 1331424000,[] => UTC))),[] => 1,[] => Piwik_Date Object ([] => 1330905600,[] => UTC))),[] => 1,[] => ,[strPeriod] => range,[strDate] => 2012-02-28,2012-03-06,[defaultEndDate] => ,[timezone] => UTC,[today] => Piwik_Date Object ([] => 1330992000,[] => UTC)),[site] => Piwik_Site Object ([] => 1),[] => Piwik_Segment Object ([] => Piwik_SegmentExpression Object ([] => Array (),[] => Array ([0] => CID,[1] => 5),[] => Array (),[] => Array ([0] => Array ([0] => AND,[1] => log_visit.custom_var_k1 = ?),[1] => Array ([0] => ,[1] => log_visit.custom_var_v1 = ?)),[string] => customVariableName1==CID;customVariableValue1==5,[parsedSubExpressions] => Array ([0] => Array ([0] => AND,[1] => Array ([0] => log_visit.custom_var_k1,[1] => ==,[2] => CID)),[1] => Array ([0] => ,[1] => Array ([0] => log_visit.custom_var_v1,[1] => ==,[2] => 5)))),[] => Array ([0] => Array ([type] => metric,[category] => Visit,[name] => Days since first visit,[segment] => daysSinceFirstVisit,[sqlSegment] => log_visit.visitor_days_since_first),[1] => Array ([type] => metric,[category] => Visit,[name] => Days since last Ecommerce order,[segment] => daysSinceLastEcommerceOrder,[sqlSegment] => log_visit.visitor_days_since_order),[2] => Array ([type] => metric,[category] => Visit,[name] => Days since last visit,[segment] => daysSinceLastVisit,[sqlSegment] => log_visit.visitor_days_since_last),[3] => Array ([type] => metric,[category] => Visit,[name] => Number of Actions,[segment] => actions,[sqlSegment] => log_visit.visit_total_actions),[4] => Array ([type] => metric,[category] => Visit,[name] => Number of visits,[segment] => visitCount,[sqlSegment] => log_visit.visitor_count_visits),[5] => Array ([type] => metric,[category] => Visit,[name] => Visit Duration (in seconds),[segment] => visitDuration,[sqlSegment] => log_visit.visit_total_time),[6] => Array ([type] => dimension,[category] => Visit,[name] => Browser,[segment] => browserName,[acceptedValues] => FF, IE, CH, SF, OP, etc.,[sqlSegment] => log_visit.config_browser_name,[sqlFilter] => ),[7] => Array ([type] => dimension,[category] => Visit,[name] => Browser version,[segment] => browserVersion,[acceptedValues] => 1.0, 8.0, etc.,[sqlSegment] => log_visit.config_browser_version,[sqlFilter] => ),[8] => Array ([type] => dimension,[category] => Visit,[name] => Continent,[segment] => continent,[sqlSegment] => log_visit.location_continent,[acceptedValues] => eur, asi, amc, amn, ams, afr, ant, oce),[9] => Array ([type] => dimension,[category] => Visit,[name] => Country,[segment] => country,[sqlSegment] => log_visit.location_country,[acceptedValues] => de, us, fr, in, es, etc.),[10] => Array ([type] => dimension,[category] => Visit,[name] => Local time,[segment] => visitLocalHour,[sqlSegment] => HOUR(log_visit.visitor_localtime),[acceptedValues] => 0, 1, 2, 3, …, 20, 21, 22, 23),[11] => Array ([type] => dimension,[category] => Visit,[name] => Operating system,[segment] => operatingSystem,[acceptedValues] => WXP, WI7, MAC, LIN, AND, IPD, etc.,[sqlSegment] => log_visit.config_os,[sqlFilter] => ),[12] => Array ([type] => dimension,[category] => Visit,[name] => Provider,[segment] => provider,[acceptedValues] => comcast.net, proxad.net, etc.,[sqlSegment] => log_visit.location_provider),[13] => Array ([type] => dimension,[category] => Visit,[name] => Resolution,[segment] => resolution,[acceptedValues] => 1280x1024, 800x600, etc.,[sqlSegment] => log_visit.config_resolution,[sqlFilter] => ),[14] => Array ([type] => dimension,[category] => Visit,[name] => Server time,[segment] => visitServerHour,[sqlSegment] => HOUR(log_visit.visit_last_action_time),[acceptedValues] => 0, 1, 2, 3, …, 20, 21, 22, 23),[15] => Array ([type] => dimension,[category] => Visit,[name] => Visit Ecommerce status at the end of the visit. For example, to select all visits that have made an Ecommerce order, the API request would contain “&segment=visitEcommerceStatus==ordered,visitEcommerceStatus==orderedThenAbandonedCart”,[segment] => visitEcommerceStatus,[acceptedValues] => none, ordered, abandonedCart, orderedThenAbandonedCart,[sqlSegment] => log_visit.visit_goal_buyer,[sqlFilter] => Array ([0] => Piwik_API_API,[1] => getVisitEcommerceStatus)),[16] => Array ([type] => dimension,[category] => Visit,[name] => Visit converted a specific Goal Id,[segment] => visitConvertedGoalId,[sqlSegment] => log_conversion.idgoal,[acceptedValues] => 1, 2, 3, etc.),[17] => Array ([type] => dimension,[category] => Visit,[name] => Visit converted at least one Goal,[segment] => visitConverted,[acceptedValues] => 0, 1,[sqlSegment] => log_visit.visit_goal_converted),[18] => Array ([type] => dimension,[category] => Visit,[name] => Visitor ID,[segment] => visitorId,[acceptedValues] => 34c31e04394bdc63 - any 16 Hexadecimal chars ID, which can be fetched using the Tracking API function getVisitorId(),[sqlSegment] => log_visit.idvisitor,[sqlFilter] => Array ([0] => Piwik_Common,[1] => convertVisitorIdToBin)),[19] => Array ([type] => dimension,[category] => Visit,[name] => Visitor IP,[segment] => visitIp,[acceptedValues] => 13.54.122.1, etc.,[sqlSegment] => log_visit.location_ip,[sqlFilter] => Array ([0] => Piwik_IP,[1] => P2N),[permission] => ),[20] => Array ([type] => dimension,[category] => Visit,[name] => Visitor type. For example, to select all visitors who have returned to the website, including those who have bought something in their previous visits, the API request would contain “&segment=visitorType==returning,visitorType==returningCustomer”,[segment] => visitorType,[acceptedValues] => new, returning, returningCustomer,[sqlSegment] => log_visit.visitor_returning,[sqlFilter] => ’


(Matthieu Aubry) #11

What is your REST request (without token_auth)?


#12

http://localhost/piwik/index.php?module=API&method=CustomVariables.getCustomVariables&idSite=1&period=range&date=2012-02-28,2012-03-06&format=JSON&token_auth=anonymous&expanded=1&segment=customVariableName1==ABC;customVariableValue1==5


(Matthieu Aubry) #13

What is the timeout in seconds in Settings > General settings?

Can you try increase it to 1800 for example? do you see the error also with a higher value?


#14

Just to confirm it is - Reports for today (or any other Date Range including today) will be processed at most every - changing from 10 seconds to 1800 seconds.

I don’t see it happen. Tried refreshing the call several times, but haven’t seen it happening. As soon as I changed it to 10 seconds I see the issue happening again.


(Matthieu Aubry) #15

OK that is a “known limitation” of Piwik. Even only 30s or 180s should be enough.


#16

I think I would recommend then to make it default to 30s and put a note on that page about the limitation and minimum value recommended at least until limitation is still valid. Thanks for your immediate help.


#17

I get the exact same exception thrown - ‘Exception’ with message ‘The requested table (id = 4) couldn’t be found in the DataTable Manager’ in /var/www/piwik/core/DataTable/Manager.php:76

Piwik is 1.7.1

I don’t think the solution is to increase the archive rebuild to 30 seconds. Regardless of the length of time you set the ‘do not rebuild the archive’ timing to, I can reproduce this issue. The problem happens every time the archive is prepared from scratch to satisfy a query, and not reused it. If the archive doesn’t have to be prepared from scratch then the operation runs successfully. So increasing the number of seconds in this setting makes the issue appear to go away, but really it just happens less often.

I can produce the error by issuing an API call to the Custom Variable table while filtering on a Custom Variable. This is my request (minus the auth token and ip) -

xx.xx.xx.xx/piwik/index.php?module=Widgetize&action=iframe&filter_limit=10&moduleToWidgetize=CustomVariables&actionToWidgetize=getCustomVariables&idSite=1&period=year&date=2012-04-02&disableLink=1&widget=1&token_auth=xxxxxxxxxxx&segment=customVariableName2==Customer;customVariableValue2==XXXX


#18

Just some more info on this - I have two separate installs of 1.7.1 and it occurs on both. Also, it seems to only occur with Custom Variables and period = range or year. It doesn’t occur for month/week/day.

owen


#19

I managed to reproduce this bug on the demo site. Using the following URL, when the archive has to be generated from scratch to satisfy the query, the page displays no data; however, when you reload the URL it displays the data you would expect.

http://demo.piwik.org/index.php?module=Widgetize&action=iframe&filter_limit=10&moduleToWidgetize=CustomVariables&actionToWidgetize=getCustomVariables&idSite=7&period=year&date=2012-04-03&disableLink=1&widget=1&segment=customVariableName3==Forum%20status;customVariableValue3==LoggedIn%20user

I will open a bug for this.

owen


#20

Bug filed