[Carbon-dev] Maximum file size to upload to G-Reg

Sanjiva Weerawarana sanjiva at wso2.com
Wed Jan 5 02:33:27 PST 2011


+1 .. I think we need to design the file storage component we're going to do
for filebus properly and use that here too to store stuff bigger than some
size on the filesystem and use the filebus features to manage/scale that
side.

Sanjiva.

On Wed, Jan 5, 2011 at 3:52 PM, Paul Fremantle <paul at wso2.com> wrote:

> I can't think it is a good idea to store Gb size data in our Registry. I
> would think you would be better storing the files in a filesystem and using
> the Registry to handle the metadata.
>
> Paul
>
>
> On 3 January 2011 08:48, Senaka Fernando <senaka at wso2.com> wrote:
>
>>
>>
>> On Mon, Jan 3, 2011 at 11:46 AM, Kishanthan Thangarajah <
>> kshanth2101 at gmail.com> wrote:
>>
>>> Hi Senaka,
>>> I tried several times increasing the memory size, but i cant upload a
>>> file with size of 700mb to registry. Always getting the java memory(heap
>>> size) error.
>>> My ram is 3GB,
>>> There are several parameters i find in wrapper.conf.
>>> -Xms256m
>>> -Xmx512m
>>> -XX:MaxPermSize=128m
>>>
>>> # Initial Java Heap Size (in MB)
>>> wrapper.java.initmemory=256
>>> # Maximum Java Heap Size (in MB)
>>> wrapper.java.maxmemory=512
>>>
>>> what i did is that i incresed the maxmemory to 1024 and MaxPermSize to
>>> 512m.
>>> Do i need to change anything else?
>>>
>>
>> No, AFAIU, you've done it correctly, but you'll need more memory. For
>> example, a PermSize of 128M can handle 100MB, for 700MB, I believe that
>> you'll need a PermSize of around 900MB. You will also need to increase the
>> other memory parameters accordingly.
>>
>> Thanks,
>> Senaka.
>>
>>>
>>> Thanks,
>>>
>>> On Thu, Dec 30, 2010 at 10:27 PM, shelan Perera <shelanrc at gmail.com>wrote:
>>>
>>>> Hi Senaka,
>>>>
>>>> File size may vary few Giga bytes (10-100 with realistic limits).
>>>> Actually it is an attempt to create a big storage aggregating few
>>>> G-Regs through a peer to peer network.I need to have a persistent storage
>>>> attached to G-Reg ( Such as a file system). Would G-Reg be efficient in
>>>> handling or have the feasibility.
>>>>
>>>>
>>>> On Thu, Dec 30, 2010 at 10:01 PM, Senaka Fernando <senaka at wso2.com>wrote:
>>>>
>>>>> Hi Shelan,
>>>>>
>>>>> When you say large chunks how big are these?
>>>>>
>>>>> The maximum file upload size varies with the maximum amount of memory
>>>>> available. Also, you need to edit a parameter on the carbon.xml file to
>>>>> increase the default file upload size of 100MB, and the wso2server.sh (or
>>>>> wrapper.conf) to increase the default allocation of memory (which is 512MB
>>>>> with 128MB permanent generation space).
>>>>>
>>>>> Thanks,
>>>>> Senaka.
>>>>>
>>>>> On Thu, Dec 30, 2010 at 9:12 PM, shelan Perera <shelanrc at gmail.com>wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I have an requirement to upload large chunks of data to G-Reg.
>>>>>> What is the maximum file upload size G-Reg can handle if I upload
>>>>>> remotely to a server.?
>>>>>>
>>>>>> I intend to use WSO2 G-Reg as a meta data management layer at the
>>>>>> primary level but would like to know is it capable of handling large data
>>>>>> chunks to upload through G-Reg directly.
>>>>>> Could you please enlighten me on the suitability of G-Reg for the
>>>>>> above requirement.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> --
>>>>>> Shelan Perera
>>>>>>
>>>>>> Blog   : http://www.shelanlk.com
>>>>>> Twitter:shelan
>>>>>> skype  :shelan.perera
>>>>>> gtalk   :shelanrc
>>>>>>
>>>>>>  I am the master of my fate:
>>>>>>  I am the captain of my soul.
>>>>>>          *invictus*
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Carbon-dev mailing list
>>>>>> Carbon-dev at wso2.org
>>>>>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Senaka Fernando*
>>>>> Associate Technical Lead & Product Manager - WSO2 G-Reg;
>>>>> WSO2, Inc.; http://wso2.com*
>>>>> Member; Apache Software Foundation; http://apache.org
>>>>>
>>>>> E-mail: senaka AT wso2.com
>>>>> **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818
>>>>> Linked-In: http://www.linkedin.com/in/senakafernando
>>>>>
>>>>> *Lean . Enterprise . Middleware
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Carbon-dev mailing list
>>>>> Carbon-dev at wso2.org
>>>>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Shelan Perera
>>>>
>>>> Blog   : http://www.shelan.info
>>>> Twitter:shelan
>>>> skype  :shelan.perera
>>>> gtalk   :shelanrc
>>>>
>>>>  I am the master of my fate:
>>>>  I am the captain of my soul.
>>>>          *invictus*
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Carbon-dev mailing list
>>>> Carbon-dev at wso2.org
>>>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Kishanthan.T
>>> University of Moratuwa.
>>> SriLanka.
>>>
>>>
>>
>>
>> --
>> *Senaka Fernando*
>> Associate Technical Lead & Product Manager - WSO2 G-Reg;
>> WSO2, Inc.; http://wso2.com*
>> Member; Apache Software Foundation; http://apache.org
>>
>> E-mail: senaka AT wso2.com
>> **P: +1 408 754 7388; ext: 51736*; *M: +94 77 322 1818
>> Linked-In: http://www.linkedin.com/in/senakafernando
>>
>> *Lean . Enterprise . Middleware
>>
>>
>> _______________________________________________
>> Carbon-dev mailing list
>> Carbon-dev at wso2.org
>> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev
>>
>>
>
>
> --
> Paul Fremantle
> CTO and Co-Founder, WSO2
> OASIS WS-RX TC Co-chair, VP, Apache Synapse
>
> Office: +44 844 484 8143
> Cell: +44 798 447 4618
>
> blog: http://pzf.fremantle.org
> twitter.com/pzfreo
> paul at wso2.com
>
> wso2.com Lean Enterprise Middleware
>
> Disclaimer: This communication may contain privileged or other confidential
> information and is intended exclusively for the addressee/s. If you are not
> the intended recipient/s, or believe that you may have received this
> communication in error, please reply to the sender indicating that fact and
> delete the copy you received and in addition, you should not print, copy,
> retransmit, disseminate, or otherwise use the information contained in this
> communication. Internet communications cannot be guaranteed to be timely,
> secure, error or virus-free. The sender does not accept liability for any
> errors or omissions.
>
> _______________________________________________
> Carbon-dev mailing list
> Carbon-dev at wso2.org
> https://wso2.org/cgi-bin/mailman/listinfo/carbon-dev
>
>


-- 
Sanjiva Weerawarana, Ph.D.
Founder, Chairman & CEO; WSO2, Inc.;  http://wso2.com/
email: sanjiva at wso2.com; phone: +94 11 763 9614; cell: +94 77 787 6880 | +1
650 265 8311
blog: http://sanjiva.weerawarana.org/

Lean . Enterprise . Middleware
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://wso2.org/pipermail/carbon-dev/attachments/20110105/85d3e0f4/attachment.htm>


More information about the Carbon-dev mailing list