linux - Java MappedByteBuffer performance getting worse and worse when you change the map size continuously -

recently had tests java mappedbytebuffer. found if continuously map same file , read it, time spend in reading getting longer , longer. if didn't change map size, faster use same map size in map size variation test.

i hava file "datafile" in 1gb filled integers.

private final file datafile = new file("~/testfile"); private final int intnum = 1024 * 1024 * 1024 / 4; // 1gb integers  @test public void writefile() throws exception {     dataoutputstream dos = new dataoutputstream(new bufferedoutputstream(new fileoutputstream(datafile)));     (int = 0; < intnum; i++) {         dos.writeint(randomutils.nextint());     }     dos.close(); }  

and method reading it

// read datafile in loop fixed map size private void buffersizeperformancetest(final int buffsize) throws exception {     stopwatch stopwatch = stopwatch.createstarted();     filechannel fc = new randomaccessfile(datafile, "r").getchannel();     mappedbytebuffer buffer;     final int readperloop = buffsize / 4;     int currentlen = 0;     int readcount = 0;     (int = 1; ; i++) {         int i1 = * buffsize;         if (i1 >= datafile.length()) {             buffer = fc.map(filechannel.mapmode.read_only, currentlen, datafile.length() - currentlen);             (int j = 0; j < readperloop; j++) {                 buffer.getint();                 readcount++;             }             break;         } else {             buffer = fc.map(filechannel.mapmode.read_only, currentlen, buffsize);             currentlen = i1;         }          (int j = 0; j < readperloop; j++) {             buffer.getint();             readcount++;         }     }     fc.close(); //        bytebufferutil.releasebytebuffer(buffer); //        system.gc();     system.out.println("readcount : " + readcount + " raf buffer size " + getmbytes(buffsize) + " mb : " + stopwatch.elapsed(timeunit.milliseconds)); } 

the variation buffsize test

private static int getmbytes(int bytes) {     return bytes / 1024 / 1024; }  // power of 2 n private static int getm(int n) {     return (int) (math.log10(n) / math.log10(2)); }  @test public void testbuffsizereadperformance() throws exception {     system.out.println(managementfactory.getruntimemxbean().getname());     (int = 0; <= getm(1024); i++) {         thread.sleep(1000);         buffersizeperformancetest((int) (math.pow(2, i) * 1024 * 1024));     } } 

variation outputs:

14071@liuzhmacbook.local readcount : 268435456 raf buffer size 1 mb : 122 readcount : 268435456 raf buffer size 2 mb : 133 readcount : 268435456 raf buffer size 4 mb : 29 readcount : 268435456 raf buffer size 8 mb : 35 readcount : 268435456 raf buffer size 16 mb : 38 readcount : 268435456 raf buffer size 32 mb : 124 readcount : 268435456 raf buffer size 64 mb : 241 readcount : 268435456 raf buffer size 128 mb : 456 readcount : 268435456 raf buffer size 256 mb : 1086 readcount : 268435456 raf buffer size 512 mb : 2458 readcount : 268435456 raf buffer size 1024 mb : 4952 

the fixed buffsize test:

@test public void testbuffsizereadperformance2() throws exception {     system.out.println(managementfactory.getruntimemxbean().getname());      (int = 0; < 10; i++) {         buffersizeperformancetest(1024 * 1024 * 1024);     } } 

output

14157@liuzhmacbook.local readcount : 268435456 raf buffer size 1024 mb : 127 readcount : 268435456 raf buffer size 1024 mb : 111 readcount : 268435456 raf buffer size 1024 mb : 20 readcount : 268435456 raf buffer size 1024 mb : 17 readcount : 268435456 raf buffer size 1024 mb : 23 readcount : 268435456 raf buffer size 1024 mb : 19 readcount : 268435456 raf buffer size 1024 mb : 21 readcount : 268435456 raf buffer size 1024 mb : 22 readcount : 268435456 raf buffer size 1024 mb : 20 readcount : 268435456 raf buffer size 1024 mb : 33 

as 2 tests shows, time spent in reading same buffsize(1024mb) quite different in 2 tests. test fixed buffsize faster variation test.

my question is: 1. how happened, why faster? 2. mappedbytebuffer occupy physical memory? see in activitymonitor, not occupy physical memory.

thanks

----- update -----

the release buffer code:

public static void releasebytebuffer(bytebuffer buffer) throws nosuchfieldexception, illegalaccessexception {     cleaner cleaner = ((directbuffer) buffer).cleaner();     cleaner.clean(); } 

i don't think cause of problem memory usage. because has same output if turn on release code , gc code. anyway, if memory usage, set loop times 100 in second test, should use more memory first test, it's faster first one.

----- update 2 -----

if turn buffsize decrease instead of increase in test 1, problem disappear.

@test public void testbuffsizereadperformance3() throws exception {     system.out.println(managementfactory.getruntimemxbean().getname());     (int = getm(1024); >= 0; i--) {         buffersizeperformancetest((int) (math.pow(2, i) * 1024 * 1024));     } } 

output:

16651@liuzhmacbook.local readcount : 268435456 raf buffer size 1024 mb : 101 readcount : 268435456 raf buffer size 512 mb : 187 readcount : 268435456 raf buffer size 256 mb : 31 readcount : 268435456 raf buffer size 128 mb : 30 readcount : 268435456 raf buffer size 64 mb : 36 readcount : 268435456 raf buffer size 32 mb : 37 readcount : 268435456 raf buffer size 16 mb : 37 readcount : 268435456 raf buffer size 8 mb : 32 readcount : 268435456 raf buffer size 4 mb : 44 readcount : 268435456 raf buffer size 2 mb : 34 readcount : 268435456 raf buffer size 1 mb : 55 

you aren't 'changing map size continuously'.you keep creating new maps, , there no mechanism whereby mappings released, including gc, using more , more memory.

you should aim use few mappedbytebuffers possible, may mean need make sizes larger well.

i don't know bytebufferutil.releasebytebuffer(buffer) does, or comes from, these things of nature cannot reliable.