Recently in oracle experiences Category

Some lob internal

| No Comments

这篇文章源于我昨天和今天跟老熊的无数次电话交流,老熊是极其强悍的,每次我跟他讨论完问题后,我对某个知识点的认识都能得到质的提高,再次感谢老熊!

 

1402e里这么提到----When a LOB is stored out-of-line in an "enable storage in row" LOB column between 36 and 84 bytes of control data remain in-line in the row piece.  这里的control data到底是指什么?

答:这里的control data就是指的12chunk的起始RDBA

col  0: [ 2]  c1 02

col  1: [ 7]  63 75 69 68 75 61 31

col  2: [84]

 00 54 00 01 01 0c 00 00 00 01 00 00 00 01 00 00 00 01 1e b9 00 40 05 00 00

 00 00 36 1a d8 00 00 00 00 00 02 01 00 01 ad 01 00 02 75 01 00 02 7d 01 00

 02 85 01 00 03 0d 01 00 03 15 01 00 03 1d 01 00 03 25 01 00 03 2d 01 00 03

 35 01 00 03 3d 01 00 03 45

这里的lob id00 00 00 01 00 00 00 01 1e b9,这里的第36byte到第84byte就是12RDBA,第1RDBA01 00 01 ad,......,第12RDBA01 00 03 45,注意这里记录的RDBA是以chunk为单位的起始RDBA,比如我这里的chunk size32K(库的block size8K),所以01 00 01 ad, 01 00 01 ae, 01 00 01 af, 01 00 01 b0都属于第一个chunk,即它们的page number都是0,但因为这四个block是连续的,且属于同一个chunk,所以oracle这里只需记录一个起始RDBA就可以了。

 

2lob index的结构到底是怎样的?

答:lob index的结构类似于unique index的结构,它长度为50byte,具体组成如下:

行头(1byte)+ lock byte1byte)+ lob index key ( 32byte,要么由16bytelob inode+4RDBA组成,要么由8RDBA组成) lob id的长度(1byte)+ lob id10byte)+ page number的长度(1byte)+ page number4byte

所以一共是1+1+32+1+10+1+4=50byte

 

3lob index损坏后如何恢复?

答:DUL10号称是支持lob index损坏后的恢复的,但这里有一个最大的问题就是假如是disable storage in row,那么lob inode会存储在page number0lob index里,假如这个page number0lob index损坏了,那么我们就无法知道最后一个page里到底有多少byte是有效的

col  0: [ 2]  c1 02

col  1: [ 7]  63 75 69 68 75 61 31

col  2: [84]

 00 54 00 01 01 0c 00 00 00 01 00 00 00 01 00 00 00 01 1e b9 00 40 05 00 00

 00 00 36 1a d8 00 00 00 00 00 02 01 00 01 ad 01 00 02 75 01 00 02 7d 01 00

 02 85 01 00 03 0d 01 00 03 15 01 00 03 1d 01 00 03 25 01 00 03 2d 01 00 03

 35 01 00 03 3d 01 00 03 45

比如上述例子里,我们知道最后一个lob page里有效的byte1a d8

SQL> select to_number('1ad8','XXXXXXXX') from dual;

 

TO_NUMBER('1AD8','XXXXXXXX')

----------------------------

                        6872

 

再来看相应的lob index

row#0[7986] flag: ------, lock: 2, len=50, data:(32):

 01 00 03 4d 01 00 03 55 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

 00 00 00 00 00 00 00

col 0; len 10; (10):  00 00 00 01 00 00 00 01 1e b9

col 1; len 4; (4):  00 00 00 0c

 

这里可以看到,最后一个chunk的起始RDBA01 00 03 55但是朋友们千万不要以为这个就是那6872byte所在的block,实际上,我dump后发现01 00 03 55里是个空的lob块(实际上这里也是一个full块,只不过所有的byte都是0x00,看起来就跟空块一样),真正的存储那6872byteblock01 00 03 57,而且我通过比对我insert的图片的最后几个byte01 00 03 57中存储的内容,算出来刚好里面的有效的byte6872

SQL> select 286*24+8 from dual;

 

  286*24+8

----------

      6872

 

另外,这里也可以这么定位最后一个lob块:

1da8(即6872)前面的那4byte0x00000036,表示这个LOB  full block的块数,注意是块数,而不是chunk数。这里0x00000036=54个块,表示有54个块是满的,第55个不是满的,里面有6872字节。

 

再看下面那个lob indexpage number0x0c,即12开始,从0开始算的。那么这个chunk里面,之前已经有12*4=48个块是满的,这里01 00 03 4d开始的chunk4个块也是满的,48+4=52,所以这个以01 00 03 55开头的chunk,有2个块是满的,就是01 00 03 55 01 00 03 56,最后那个块我之前说的01 00 03 57了。

 

这就提示我们,当极端的情况下(disable storage in rowlob inode所在的lob index已经损坏),则我们需要扫描lob segment,取出其中存储的lob idpage number(相当于重构lob index),但是这里在扫描某个data object id所属的lob idpage number的时候,需要额外处理同一个lob id下的最大的page number----要去扫描这个page number下的所有block,且最大的RDBA里最后一个不为0byte就是lob inode记录的最后一个page的有效byte数。

Lob的字符集和字节序

| No Comments

402e里这么描述CLOBNCLOB所用的字符集:

CLOB is used to store character data using the database character set. The internal

code is 112.

NCLOB is used to store character data using the national character set. The internal

code is 112.

 

但这是不对的,至少是不全面的,我们来看一下真相是什么。

对于定长的字符集而言,如WE8MSWIN1252 , AR8ISO8859P6等,确实就如402e里所描述的那样,CLOB所用的字符集就是和库里所用的字符集一致。

 

对于变长的字符集而言,如ZHS16GBKUTF8AL32UTF8等,oracle这里实际上会做一个字符集间的转换,也就是说,实际存储在CLOB/NCLOB里的字符集其实是internal fixed-width Unicode character set这个internal fixed-width Unicode character set目前来看,只能是UCS2 或者AL16UTF16

 

UCS2AL16UTF16的大致区别为:

* UCS2 is a subset of the UTF16 standard.

* UCS2的字节序跟库所在的platform的字节序一致

* AL16UTF16的字节序始终是big endian

从这里我们可以推断出,在big endianplatformUCS2就等于AL16UTF16

 

如下的这张图非常经典,已经说明了一切:

 

lob_internal_characterset.JPG 

关于这个图的详细信息大家可以去看"CLOBs and NCLOBs character set storage in Oracle Release 8i, 9i, 10g and 11g. [ID 257772.1]"里面解释的非常清楚。

 

里面有一段话这么说:

On Big Endian platforms, when a database with a varying-width database or national character set is migrated to Oracle Database 10g, no conversion of the stored CLOB and NCLOB data is necessary as UCS2 and AL16UTF16 are equivalent. On Little Endian platforms, the storage character set is not changed as part of the upgrade. Only column meta-data is updated to indicate that the CLOB/NCLOB columns are in Little Endian format. There is no usage impact after the migration because the internal format is not exposed to users, and proper conversions will happen implicitly in the system.

 

这里的meta-data是指lob$里的字段propertysql.bsq里明确指出:

property      number not null,           /* 0x00 = user defined lob column */

                                    /* 0x01 = kernel column(s) stored as lob */

                                     /* 0x02 = user lob column with row data */

                                            /* 0x04 = partitioned LOB column */

                                   /* 0x0008 = LOB In Global Temporary Table */

                                          /* 0x0010 = Session-specific table */

                                      /* 0x0020 = lob with compressed header */

                                        /* 0x0040 = lob using shared segment */

                                  /* 0x0080 = first lob using shared segment */

                                   /* 0x0100 = klob and inline image coexist */

                                /* 0x0200 = LOB data in little endian format */

 

最后我们的结论为:

1、如果库的字符集是定长字符集,则CLOB所用的字符集和库的字符集一致,字节序也和库所在的platform的字节序一致。

2、当库的字符集是变长字符集的时候,CLOB/NCLOB所用的字符集只能是UCS2或者AL16UTF16AL16UFT16始终是big endianUCS2的字节序则和库所在的platform的字节序一致。具体又分为如下这几种情况:

 1) 如果是8i/9i,则CLOB/NCLOB所用的字符集都是UCS2

 2) 如果是10gCLOB/NCLOB所用的字符集基本上都是AL16UTF16,为UCS2的只有上图中所描述的那两种情况

 

其实这篇文章只是上述那篇metalink文档的一个翻译和总结,我为什么要写这篇文章来源于我昨天跟老熊之间的一次电话交流,他纠正了我长久以来的一个错误观点并且使我对lob internal的认识已经变得异常清晰,非常感谢老熊!

Recent Comments

  • Noma Lauw: Great job for creating such one of a kind collection read more
  • Vina Pagni: Damn, cool website. I actually came across this on Ask read more
  • the diet solution program reviews: By far, one of the best post l have come read more
  • Lose Ten Pounds in Three Days: By a long shot, one of the best post l read more
  • exercises to help lose 15 pounds: Without doubt, one of the best article l have come read more
  • Lose 20 Pounds a Week: I certainly enjoy your post, but having problem subscribing to read more
  • Aiko Potsander: Please continue to keep the good work! Cheers. read more
  • 毕业论文: 学习了。 read more
  • cui hua: 不用改数据,你改row directory里的指针就可以了——这就是我文中提到的update internal。 read more
  • yangjiawei: 领导,不好意思,再请问一下 我现在遇到一个问题,我现在已经将ind$里两个索引的状态改好了,数据库也拉起来了~! 但是在修改obj$里name里为DEPENDENCY$这一行数据的data_object_id时遇到了困难,因为他原先的长度为2个字节,现在由于我move了一下,他的长度变成了4个字节,结果如下: 原先: col 1[2] @906: 0xc1 0x5d ==>92 read more

About this Archive

This page is an archive of recent entries in the oracle experiences category.

my life is the previous category.

other experiences is the next category.

Find recent content on the main index or look in the archives to find all content.