c# - Read bytea data is slow in PostgreSQL -


i store data in bytea column in postgresql 9.5 database on windows.

the data transmission speed lower expect : 1.5mb per second.

the following code

        using (var conn = connectionprovider.getopened())         using (var comm = new npgsqlcommand("select mycolumn mytable", conn))         using (var dr = comm.executereader())         {             var clock = stopwatch.startnew();             while (dr.read())             {                 var bytes = (byte[])dr[0];                 debug.writeline($"bytes={bytes.length}, time={clock.elapsed}");                 clock.restart();             }         } 

produces following output

bytes=3895534, time=00:00:02.4397086 bytes=4085257, time=00:00:02.7220734 bytes=4333460, time=00:00:02.4462513 bytes=4656500, time=00:00:02.7401579 bytes=5191876, time=00:00:02.7959250 bytes=5159785, time=00:00:02.7693224 bytes=5184718, time=00:00:03.0613514 bytes=720401, time=00:00:00.0227767 bytes=5182772, time=00:00:02.7704914 bytes=538456, time=00:00:00.2996142 bytes=246085, time=00:00:00.0003131 total: 00:00:22.5199268 

the strange thing reading last 246kb took less millisecond, , reading 720kb in middle took 22ms.

is reading speed 5mb per 3 sec normal? how can increase reading speed?

details.

my application starts postgresql server on startup , shut downs on exit.

i start server following code

public static void startserver(string datadirectory, int port) {           invoke("pg_ctl", $"start -w  -d \"{datadirectory}\" -m fast -o \"-b 512mb -p {port} -c temp_buffers=32mb -c work_mem=32mb\""); } 

also, change storage type column :

alter table mytable alter column mycolumn set storage external; 

i use npgsql 3.0.4.0 , postgresql 9.5 on windows 10

running here npgsql 3.0.8 (should same), postgresql 9.5, windows 10 , npgsql don't results @ all:

bytes=3895534, time=00:00:00.0022591 bytes=4085257, time=00:00:00.0208912 bytes=4333460, time=00:00:00.0228702 bytes=4656500, time=00:00:00.0237144 bytes=5191876, time=00:00:00.0317834 bytes=5159785, time=00:00:00.0268229 bytes=5184718, time=00:00:00.0159028 bytes=720401, time=00:00:00.0130150 bytes=5182772, time=00:00:00.0153306 bytes=538456, time=00:00:00.0021693 bytes=246085, time=00:00:00.0005174 

first, server running against, on localhost or on remote machine?

the second thing comes mind, stopping , starting server part of test. slow performance you're seeing may part of warm-up on postgresql's side. try remove , see if results can reproduced after several times of running tests.

otherwise looks environmental problem (client machine, server machine or network). i'd try reproduce problem on different machine or in different setting , go there.


Comments

Popular posts from this blog

PySide and Qt Properties: Connecting signals from Python to QML -

c# - DevExpress.Wpf.Grid.InfiniteGridSizeException was unhandled -

scala - 'wrong top statement declaration' when using slick in IntelliJ -