We have been getting this assertion failure with SQLAnywhere version 
11.0.1.2376:

    Internal database error *** ERROR *** Assertion failed: 101413 
(11.0.1.2376)

Prior to this assertion, the db server is mostly idle.  The assertion 
happens during a small time period when a bunch of connections are made to 
the database from multiple programs at the same time.  This connection 
pattern is typically for our applications and works normally for 2-3 weeks. 
Then once every 2-3 weeks, we hit upon this assertion error.

After the error, the db server terminates all connections and does not 
respond to any more connection requests.  We need to restart the database 
server.

Is this a known bug?  More technical details below:

-- Jiang Wu
   jwu@retrevo.com


DB startup param:
=================

-n star9999 -zl -zp -z -o c:\dbcrash\dbdebug.txt -os 1m -gn 80 -gm 1000 -sb 
0 -ti 0  -c 10G -cw -ch 90%%  -x tcpip{ServerPort=9999;dobroadcast=no} 
e:\wormhole1\kvsdb.db -m -n wormhole1 f:\wormhole2\kvsdb.db -m -n wormhole2 
....

OS setup:
=========

Windows Server 2003 Enterprise Edition, Service Pack 2, Physical Address 
Extension
16GB RAM, dual xenon quad core

Server side connection log shows:
=================================

>>>>>>>>>>>>>>> starting of connection activities <<<<<<<<<
I. 04/24 00:18:02. Connection from 192.168.5.17:1452 assigned connection ID 
27753
I. 04/24 00:18:02. Connection from 192.168.5.17:1453 assigned connection ID 
27754
I. 04/24 00:18:02. Connection from 192.168.5.17:1454 assigned connection ID 
27755
I. 04/24 00:18:07. Connection from 192.168.5.17:1645 assigned connection ID 
27756
I. 04/24 00:18:12. Connection from 192.168.5.17:1819 assigned connection ID 
27757
I. 04/24 00:18:41. Connection from 192.168.5.17:2631 assigned connection ID 
27758
I. 04/24 00:18:41. Connection from 192.168.5.17:2633 assigned connection ID 
27759
I. 04/24 00:18:46. Connection from 192.168.5.17:2876 assigned connection ID 
27760
I. 04/24 00:18:54. Connection from 192.168.5.17:3145 assigned connection ID 
27761
>>>>>>>>>>>>>>> First 9 connections were successful <<<<<<<<<

I. 04/24 00:19:00. TCP/IP: Received connection request from 
192.168.5.17:3319.
I. 04/24 00:19:00. Connection from 192.168.5.17:3319 assigned connection ID 
27762
I. 04/24 00:19:00. Connection ID 27762: "nova" connected to database 
"wormhole3" from SQL Anywhere 9.0.2 (3581) client over TCPIP
I. 04/24 00:19:00. Connection ID 27762: AppInfo is HOST=SHIRE;OS=Windows 
2003 Build 3790 Service Pack 
2;PID=0x9f4;THREAD=0x19dc;EXE=c:\tools\jdk16\bin\java.exe;VERSION=9.0.2.3581;API=ODBC;TIMEZONEADJUSTMENT=-420
I. 04/24 00:19:00. Connection ID 27762: Server character set conversion will 
occur
    database character set is "UTF-8"
    using client character set "windows-1252"
E. 04/24 00:21:47. *** ERROR *** Assertion failed: 101413 (11.0.1.2376)
Unable to allocate a multi-page block of memory
....
>>>>>>>>>>>>>>> After this, every connections were terminated <<<<<<<<<<<<<
I. 04/24 00:21:47. Connection ID 27760: connection "nova" terminated 
abnormally
I. 04/24 00:21:47. Connection ID 27754: connection "nova" terminated 
abnormally
I. 04/24 00:21:47. Connection ID 27756: connection "nova" terminated 
abnormally
I. 04/24 00:21:47. Connection ID 27761: connection "nova" terminated 
abnormally
....
 

Hi Jiang,

This assertion indicates that the database server has run out of address 
space while trying to perform an allocation outside of the database page 
cache. By default, the server reserves 512mb of address space for 
non-cache operations. This may not be enough for all systems. Since you 
are using AWE, you can increase the amount of address space reserved by 
the server by using the -cm server option. See:

http://dcx.sybase.com/index.html#1101en/dbadmin_en11/dbserver-s-4740598.html

How large is your database file? Do you use encryption or compressed 
columns? Have you considered moving to a 64-bit server?

  -Nathan

Nathan Auch
Sybase iAnywhere

On 04/24/2010 02:06 PM, Jiang Wu wrote:
> We have been getting this assertion failure with SQLAnywhere version
> 11.0.1.2376:
>
> Internal database error *** ERROR *** Assertion failed: 101413
> (11.0.1.2376)
>
> Prior to this assertion, the db server is mostly idle. The assertion
> happens during a small time period when a bunch of connections are made
> to the database from multiple programs at the same time. This connection
> pattern is typically for our applications and works normally for 2-3
> weeks. Then once every 2-3 weeks, we hit upon this assertion error.
>
> After the error, the db server terminates all connections and does not
> respond to any more connection requests. We need to restart the database
> server.
>
> Is this a known bug? More technical details below:
>
> -- Jiang Wu
> jwu@retrevo.com
>
>
> DB startup param:
> =================
>
> -n star9999 -zl -zp -z -o c:\dbcrash\dbdebug.txt -os 1m -gn 80 -gm 1000
> -sb 0 -ti 0 -c 10G -cw -ch 90%% -x tcpip{ServerPort=9999;dobroadcast=no}
> e:\wormhole1\kvsdb.db -m -n wormhole1 f:\wormhole2\kvsdb.db -m -n
> wormhole2 ...
>
> OS setup:
> =========
>
> Windows Server 2003 Enterprise Edition, Service Pack 2, Physical Address
> Extension
> 16GB RAM, dual xenon quad core
>
> Server side connection log shows:
> =================================
>
>>>>>>>>>>>>>>>> starting of connection activities <<<<<<<<<
> I. 04/24 00:18:02. Connection from 192.168.5.17:1452 assigned connection
> ID 27753
> I. 04/24 00:18:02. Connection from 192.168.5.17:1453 assigned connection
> ID 27754
> I. 04/24 00:18:02. Connection from 192.168.5.17:1454 assigned connection
> ID 27755
> I. 04/24 00:18:07. Connection from 192.168.5.17:1645 assigned connection
> ID 27756
> I. 04/24 00:18:12. Connection from 192.168.5.17:1819 assigned connection
> ID 27757
> I. 04/24 00:18:41. Connection from 192.168.5.17:2631 assigned connection
> ID 27758
> I. 04/24 00:18:41. Connection from 192.168.5.17:2633 assigned connection
> ID 27759
> I. 04/24 00:18:46. Connection from 192.168.5.17:2876 assigned connection
> ID 27760
> I. 04/24 00:18:54. Connection from 192.168.5.17:3145 assigned connection
> ID 27761
>>>>>>>>>>>>>>>> First 9 connections were successful <<<<<<<<<
>
> I. 04/24 00:19:00. TCP/IP: Received connection request from
> 192.168.5.17:3319.
> I. 04/24 00:19:00. Connection from 192.168.5.17:3319 assigned connection
> ID 27762
> I. 04/24 00:19:00. Connection ID 27762: "nova" connected to database
> "wormhole3" from SQL Anywhere 9.0.2 (3581) client over TCPIP
> I. 04/24 00:19:00. Connection ID 27762: AppInfo is HOST=SHIRE;OS=Windows
> 2003 Build 3790 Service Pack
> 2;PID=0x9f4;THREAD=0x19dc;EXE=c:\tools\jdk16\bin\java.exe;VERSION=9.0.2.3581;API=ODBC;TIMEZONEADJUSTMENT=-420
>
> I. 04/24 00:19:00. Connection ID 27762: Server character set conversion
> will occur
> database character set is "UTF-8"
> using client character set "windows-1252"
> E. 04/24 00:21:47. *** ERROR *** Assertion failed: 101413 (11.0.1.2376)
> Unable to allocate a multi-page block of memory
> ...
>>>>>>>>>>>>>>>> After this, every connections were terminated <<<<<<<<<<<<<
> I. 04/24 00:21:47. Connection ID 27760: connection "nova" terminated
> abnormally
> I. 04/24 00:21:47. Connection ID 27754: connection "nova" terminated
> abnormally
> I. 04/24 00:21:47. Connection ID 27756: connection "nova" terminated
> abnormally
> I. 04/24 00:21:47. Connection ID 27761: connection "nova" terminated
> abnormally
> ...
>
>

DB files are about 150GB each.  But there are several databases totaling 
about 1.5 TB of data.  We don't have any encryption or compressed data.

We will give the -cm option a try and report back on the results.  As for 
64-bit server, we will recommend it to the IT folks.  Thanks.

-- Jiang Wu
   jwu@retrevo.com


"Nathan Auch (Sybase)" <nauch@sybase.com> wrote in message 
news:4bd5b481$1@forums-1-dub...
> Hi Jiang,
>
> This assertion indicates that the database server has run out of address 
> space while trying to perform an allocation outside of the database page 
> cache. By default, the server reserves 512mb of address space for 
> non-cache operations. This may not be enough for all systems. Since you 
> are using AWE, you can increase the amount of address space reserved by 
> the server by using the -cm server option. See:
>
> http://dcx.sybase.com/index.html#1101en/dbadmin_en11/dbserver-s-4740598.html
>
> How large is your database file? Do you use encryption or compressed 
> columns? Have you considered moving to a 64-bit server?
>
>  -Nathan
>
> Nathan Auch
> Sybase iAnywhere
>
> On 04/24/2010 02:06 PM, Jiang Wu wrote:
>> We have been getting this assertion failure with SQLAnywhere version
>> 11.0.1.2376:
>>
>> Internal database error *** ERROR *** Assertion failed: 101413
>> (11.0.1.2376)
>>
>> Prior to this assertion, the db server is mostly idle. The assertion
>> happens during a small time period when a bunch of connections are made
>> to the database from multiple programs at the same time. This connection
>> pattern is typically for our applications and works normally for 2-3
>> weeks. Then once every 2-3 weeks, we hit upon this assertion error.
>>
>> After the error, the db server terminates all connections and does not
>> respond to any more connection requests. We need to restart the database
>> server.
>>
>> Is this a known bug? More technical details below:
>>
>> -- Jiang Wu
>> jwu@retrevo.com
>>
>>
>> DB startup param:
>> =================
>>
>> -n star9999 -zl -zp -z -o c:\dbcrash\dbdebug.txt -os 1m -gn 80 -gm 1000
>> -sb 0 -ti 0 -c 10G -cw -ch 90%% -x tcpip{ServerPort=9999;dobroadcast=no}
>> e:\wormhole1\kvsdb.db -m -n wormhole1 f:\wormhole2\kvsdb.db -m -n
>> wormhole2 ...
>>
>> OS setup:
>> =========
>>
>> Windows Server 2003 Enterprise Edition, Service Pack 2, Physical Address
>> Extension
>> 16GB RAM, dual xenon quad core
>>
>> Server side connection log shows:
>> =================================
>>
>>>>>>>>>>>>>>>>> starting of connection activities <<<<<<<<<
>> I. 04/24 00:18:02. Connection from 192.168.5.17:1452 assigned connection
>> ID 27753
>> I. 04/24 00:18:02. Connection from 192.168.5.17:1453 assigned connection
>> ID 27754
>> I. 04/24 00:18:02. Connection from 192.168.5.17:1454 assigned connection
>> ID 27755
>> I. 04/24 00:18:07. Connection from 192.168.5.17:1645 assigned connection
>> ID 27756
>> I. 04/24 00:18:12. Connection from 192.168.5.17:1819 assigned connection
>> ID 27757
>> I. 04/24 00:18:41. Connection from 192.168.5.17:2631 assigned connection
>> ID 27758
>> I. 04/24 00:18:41. Connection from 192.168.5.17:2633 assigned connection
>> ID 27759
>> I. 04/24 00:18:46. Connection from 192.168.5.17:2876 assigned connection
>> ID 27760
>> I. 04/24 00:18:54. Connection from 192.168.5.17:3145 assigned connection
>> ID 27761
>>>>>>>>>>>>>>>>> First 9 connections were successful <<<<<<<<<
>>
>> I. 04/24 00:19:00. TCP/IP: Received connection request from
>> 192.168.5.17:3319.
>> I. 04/24 00:19:00. Connection from 192.168.5.17:3319 assigned connection
>> ID 27762
>> I. 04/24 00:19:00. Connection ID 27762: "nova" connected to database
>> "wormhole3" from SQL Anywhere 9.0.2 (3581) client over TCPIP
>> I. 04/24 00:19:00. Connection ID 27762: AppInfo is HOST=SHIRE;OS=Windows
>> 2003 Build 3790 Service Pack
>> 2;PID=0x9f4;THREAD=0x19dc;EXE=c:\tools\jdk16\bin\java.exe;VERSION=9.0.2.3581;API=ODBC;TIMEZONEADJUSTMENT=-420
>>
>> I. 04/24 00:19:00. Connection ID 27762: Server character set conversion
>> will occur
>> database character set is "UTF-8"
>> using client character set "windows-1252"
>> E. 04/24 00:21:47. *** ERROR *** Assertion failed: 101413 (11.0.1.2376)
>> Unable to allocate a multi-page block of memory
>> ...
>>>>>>>>>>>>>>>>> After this, every connections were terminated 
>>>>>>>>>>>>>>>>> <<<<<<<<<<<<<
>> I. 04/24 00:21:47. Connection ID 27760: connection "nova" terminated
>> abnormally
>> I. 04/24 00:21:47. Connection ID 27754: connection "nova" terminated
>> abnormally
>> I. 04/24 00:21:47. Connection ID 27756: connection "nova" terminated
>> abnormally
>> I. 04/24 00:21:47. Connection ID 27761: connection "nova" terminated
>> abnormally
>> ...
>>
>>
> 

Just to confirm that increasing the memory as specified using -cm solved our 
problem.  Thank you for your help.

-- Jiang Wu
   jwu@retrevo.com

"Nathan Auch (Sybase)" <nauch@sybase.com> wrote in message 
news:4bd5b481$1@forums-1-dub...
> Hi Jiang,
>
> This assertion indicates that the database server has run out of address 
> space while trying to perform an allocation outside of the database page 
> cache. By default, the server reserves 512mb of address space for 
> non-cache operations. This may not be enough for all systems. Since you 
> are using AWE, you can increase the amount of address space reserved by 
> the server by using the -cm server option. See:
>
> http://dcx.sybase.com/index.html#1101en/dbadmin_en11/dbserver-s-4740598.html
>
> How large is your database file? Do you use encryption or compressed 
> columns? Have you considered moving to a 64-bit server?
>
>  -Nathan
>
> Nathan Auch
> Sybase iAnywhere
>
> On 04/24/2010 02:06 PM, Jiang Wu wrote:
>> We have been getting this assertion failure with SQLAnywhere version
>> 11.0.1.2376:
>>
>> Internal database error *** ERROR *** Assertion failed: 101413
>> (11.0.1.2376)
>>
>> Prior to this assertion, the db server is mostly idle. The assertion
>> happens during a small time period when a bunch of connections are made
>> to the database from multiple programs at the same time. This connection
>> pattern is typically for our applications and works normally for 2-3
>> weeks. Then once every 2-3 weeks, we hit upon this assertion error.
>>
>> After the error, the db server terminates all connections and does not
>> respond to any more connection requests. We need to restart the database
>> server.
>>
>> Is this a known bug? More technical details below:
>>
>> -- Jiang Wu
>> jwu@retrevo.com
>>
>>
>> DB startup param:
>> =================
>>
>> -n star9999 -zl -zp -z -o c:\dbcrash\dbdebug.txt -os 1m -gn 80 -gm 1000
>> -sb 0 -ti 0 -c 10G -cw -ch 90%% -x tcpip{ServerPort=9999;dobroadcast=no}
>> e:\wormhole1\kvsdb.db -m -n wormhole1 f:\wormhole2\kvsdb.db -m -n
>> wormhole2 ...
>>
>> OS setup:
>> =========
>>
>> Windows Server 2003 Enterprise Edition, Service Pack 2, Physical Address
>> Extension
>> 16GB RAM, dual xenon quad core
>>
>> Server side connection log shows:
>> =================================
>>
>>>>>>>>>>>>>>>>> starting of connection activities <<<<<<<<<
>> I. 04/24 00:18:02. Connection from 192.168.5.17:1452 assigned connection
>> ID 27753
>> I. 04/24 00:18:02. Connection from 192.168.5.17:1453 assigned connection
>> ID 27754
>> I. 04/24 00:18:02. Connection from 192.168.5.17:1454 assigned connection
>> ID 27755
>> I. 04/24 00:18:07. Connection from 192.168.5.17:1645 assigned connection
>> ID 27756
>> I. 04/24 00:18:12. Connection from 192.168.5.17:1819 assigned connection
>> ID 27757
>> I. 04/24 00:18:41. Connection from 192.168.5.17:2631 assigned connection
>> ID 27758
>> I. 04/24 00:18:41. Connection from 192.168.5.17:2633 assigned connection
>> ID 27759
>> I. 04/24 00:18:46. Connection from 192.168.5.17:2876 assigned connection
>> ID 27760
>> I. 04/24 00:18:54. Connection from 192.168.5.17:3145 assigned connection
>> ID 27761
>>>>>>>>>>>>>>>>> First 9 connections were successful <<<<<<<<<
>>
>> I. 04/24 00:19:00. TCP/IP: Received connection request from
>> 192.168.5.17:3319.
>> I. 04/24 00:19:00. Connection from 192.168.5.17:3319 assigned connection
>> ID 27762
>> I. 04/24 00:19:00. Connection ID 27762: "nova" connected to database
>> "wormhole3" from SQL Anywhere 9.0.2 (3581) client over TCPIP
>> I. 04/24 00:19:00. Connection ID 27762: AppInfo is HOST=SHIRE;OS=Windows
>> 2003 Build 3790 Service Pack
>> 2;PID=0x9f4;THREAD=0x19dc;EXE=c:\tools\jdk16\bin\java.exe;VERSION=9.0.2.3581;API=ODBC;TIMEZONEADJUSTMENT=-420
>>
>> I. 04/24 00:19:00. Connection ID 27762: Server character set conversion
>> will occur
>> database character set is "UTF-8"
>> using client character set "windows-1252"
>> E. 04/24 00:21:47. *** ERROR *** Assertion failed: 101413 (11.0.1.2376)
>> Unable to allocate a multi-page block of memory
>> ...
>>>>>>>>>>>>>>>>> After this, every connections were terminated 
>>>>>>>>>>>>>>>>> <<<<<<<<<<<<<
>> I. 04/24 00:21:47. Connection ID 27760: connection "nova" terminated
>> abnormally
>> I. 04/24 00:21:47. Connection ID 27754: connection "nova" terminated
>> abnormally
>> I. 04/24 00:21:47. Connection ID 27756: connection "nova" terminated
>> abnormally
>> I. 04/24 00:21:47. Connection ID 27761: connection "nova" terminated
>> abnormally
>> ...
>>
>>
>