author：skate
time：2010-05-18
同事维护一个Oracle 10.1.0.3.0的数据库时，导致ASM实例Crash，数据库切换。
这是一个启用很早的Oracle10g数据库，一直运行得很稳定，故障率很低。
查看日志，发现如下错误信息：
Wed Aug 29 12:05:24 2007
alter database datafile '+DATADG/mmsdb/datafile/gateway.295.1' resize 12000M
Wed Aug 29 12:05:25 2007
Completed: alter database datafile '+DATADG/mmsdb/datafile/ga
Wed Aug 29 12:10:14 2007
create tablespace dodd datafile size 100M
Wed Aug 29 12:10:15 2007
Starting control autobackup
Created Oracle managed file /msflsh/MMSDB/autobackup/2007_08_29/o1_mf_n_631887015_3f9wf82j_.bkp
Control autobackup written to DISK device
handle '/msflsh/MMSDB/autobackup/2007_08_29/o1_mf_n_631887015_3f9wf82j_.bkp'
Completed: create tablespace dodd datafile size 100M
Wed Aug 29 12:11:24 2007
drop tablespace dodd
Wed Aug 29 12:11:25 2007
Deleted Oracle managed file +DATADG/mmsdb/datafile/dodd.313.5
Starting control autobackup
Created Oracle managed file /msflsh/MMSDB/autobackup/2007_08_29/o1_mf_n_631887085_3f9whfv1_.bkp
Control autobackup written to DISK device
handle '/msflsh/MMSDB/autobackup/2007_08_29/o1_mf_n_631887085_3f9whfv1_.bkp'
Completed: drop tablespace dodd
Wed Aug 29 12:21:48 2007
alter database datafile '+DATADG/mmsdb/datafile/gateway.295.1' resize 15000M
Wed Aug 29 12:21:48 2007
Errors in file /opt/oracle/admin/mmsdb/bdump/mmsdb_asmb_8036.trc:
ORA-00600: internal error code, arguments: [kffmXpResize], [26], [32], [], [], [], [], []
Wed Aug 29 12:21:51 2007
Errors in file /opt/oracle/admin/mmsdb/bdump/mmsdb_asmb_8036.trc:
ORA-00600: internal error code, arguments: [kffmXpResize], [26], [32], [], [], [], [], []
Wed Aug 29 12:21:51 2007
ASMB: terminating instance due to error 486
Wed Aug 29 12:21:51 2007
Errors in file /opt/oracle/admin/mmsdb/bdump/mmsdb_pmon_7990.trc:
ORA-00486: ASMB process terminated with error
Instance terminated by ASMB, pid = 8036
注意日志，首先是Resize缩小了数据文件，然后再Resize放大了文件。
这正好完全吻合了Oracle的一个Bug：3952778
这个Bug的描述为：
Shrinking and then growing an ASM file can result in an internal error ORA-600 [kffmxpresize].
已经确定影响版本：10.1.0.3
看到没有，我们的数据库环境和同事的操作和这个Bug完全吻合，于是ASM实例Crash，数据库崩溃了。
真让人晕倒！
至于这个Bug是否百发百中，有兴趣的朋友可以试试看
这次事故给我们的教训是：DBA也需要看手气！出门最好点一炷香！！！
这次事故给DBA的教训是：如果经常手气不好，一定要找找原因
----end----