![]() ![]()
![]() In order to save information storage space and improve information transmission efficiency, a large amount of actual data must be effectively compressed. In this case, data compression is an inevitable choice. With the development of the information society, people are faced with rapidly growing information, and the pressure on storing, transmitting and processing such massive information is increasing. If there is a problem with both the main database and the backup database, data loss will be catastrophic. Blob field with compression data is not valid software#This cold backup method is obviously unrealistic the hot-backup system (autonomously developed by the specific system) can only correspond to one server due to the software reasons. Besides, 10 days are required to continuously copy the national central database (8000GB) cold backup to another server, during which the database and all services must be shut down. For example, it takes about 4 minutes for the “processing system” to remotely read the second sampling data of 6 elements in an instrument, and it takes at least 1.5 days to exchange the updated station observation data to the disciplinary center. In addition, the total data with a time resolution of minutes and seconds accounts for more than 95% (or more) of the total space of the database.Īs all the minute and second data is stored in the format of “uncompressed Clob+Ascii”, and the database is caught in problems of huge data storage space, low data exchange speed, low read-write speed, operation and maintenance difficulty. ![]() ![]() The total database size is about 8000GB, which is still increasing by 800GB every year. As of August 2018, the data outputted by 3328 sets of observation instruments (364 sets of second sampling instruments, 2126 sets of minute-sampling instruments, 838 sets of hourly and daily sampling instruments) is stored in the database. The national center is the collection center for specific data across the country and is also the largest specific database. The management system exchanges the station data to the provincial bureau, national center, and disciplinary center on daily timing ( Liu Gaochuan, 2008). The former is responsible for daily data collection and storage, while the latter is responsible for daily data preprocessing and product data calculation. Blob field with compression data is not valid Pc#There are two main software systems: management system (B/S architecture, Running on the server) and processing system (C/S architecture, Running on the client PC machine). In order to facilitate data exchange at all levels, a unified database management system (Oracle10g) and a unified database table structure are adopted nationwide ( Zhou Kechang et al, 2009, 2010 Liu Gaochuan 2008). The software system is a four-level interconnected distributed system consisting of station, provincial bureau, national center and disciplinary center. The results of this paper are of high reference significance to the LOB data storage design and software development.Īt the end of 2007, the “10th Five-Year” system of a specific network was officially completed and put into operation. The exchange rate of the second data is at least 7.89 times of the present rate, and the station data can be exchanged to the disciplinary center within 2–3 hours (currently 1.5 days) (3) The simplest and most widely used direct read method by software developers has poor database read efficiency, while the substr+threadPool technique shows higher database read efficiency no matter for Clob or Blob, for compressed or uncompressed, which brings a leap-forward improvement in the read speed of LOB data. The read-write speed is greatly improved, and the overall capacity of the database is reduced to 7% (or less). The results show that: (1) Blob is superior to Clob in terms of storage, exchange, or read-write speed (2) For the specific database, Blob+Gzip is the optimal storage structure of the minute and second data. The read speed test is performed by the direct read, substr read, and substr+threadPool read techniques. Aiming at the problems of huge storage space, low exchange speed and low read-write speed of the current specific oracle database, the read-write speed and exchange speed tests are performed on the compressed and uncompressed Clob and Blob data by three compression algorithms, including Bzip2, Gzip and GzipIO respectively. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |