Attempt to create uniform indexing on subsets of columns

Artem Mironov

Abstract


Due to the growing volume of data and the increasing variety of processing requirements, there is a shift from real-time data processing to the use of pre-saved and pre-prepared results. In some cases, DBMS address performance issues by increasing memory usage. However, it is important to explore memory-saving options while maintaining the results of methods based on approaches such as indexing, hashing, and neural algorithms. This article discusses a method for improving the efficiency of search queries in DBMS for large tables. The proposed method is based on indexing with the ability to search across specific indexed columns. It incorporates elements of clustering using a Kohonen map, as well as the preservation of additional metadata. This approach could help eliminate complex nested indexing compared to the classical B-tree. Proper implementation of this approach could allow efficient processing of tables with different search needs across various groups of columns, where maintaining indexing for each large query type or group of queries can lead to significant memory consumption and reduced performance when handling large memory blocks, the growth of which is not linear.


Full Text:

PDF (Russian)

References


Abdel-Basset M. et al. An improved nature inspired meta-heuristic algorithm for 1-D bin packing problems //Personal and Ubiquitous Computing. – 2018. – Т. 22. – №. 5-6. – С. 1117-1132.

Chamoso, Pablo, et al. "Social computing for image matching." PloS one 13.5 (2018): e0197576.

Das S. et al. Automatically indexing millions of databases in microsoft azure sql database //Proceedings of the 2019 International Conference on Management of Data. – 2019. – С. 666-679.

Dodonov A. et al. Method of Parallel Information Object Search in Unified Information Spaces //International Journal of Computer Network and Information Security (IJCNIS). – 2021. – Т. 13. – №. 4. – С. 1-13.

Gorokhovatskyi V. A., Gorokhovatskiy A. V., Peredrii Y. О. Hashing of structural descriptions at building of the class image descriptor, computing of relevance and classification of the visual objects //Telecommunications and Radio Engineering. – 2018. – Т. 77. – №. 13.

Graefe G. et al. Modern B-tree techniques //Foundations and Trends® in Databases. – 2011. – Т. 3. – №. 4. – С. 203-402.

Haynes, David, et al. "High performance analysis of big spatial data." 2015 IEEE International Conference on Big Data (Big Data). IEEE, 2015.

Iljin P. L., Munerman V. J. Recursive computation of the multidimensional matrix determinant. Systems of computerized mathematics and their appendices: XX International Scientific Conference. Smolensk: SmolSU publishing. 2019. Vol. 1, Issue 20. pp. 162-166. (In Russ).

Kirikova A., Mironov A. Using Metadata-indexing to Improve the Efficiency of Complex Operations //2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus). – IEEE, 2021. – С. 2124-2127.

Kirikova A., Mironov A., Munerman V. The Method of Composition Hash-functions for Optimize a Task of Searching Images in Dataset //2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus). – IEEE, 2020. – С. 1983-1986.

Levin N. A., Munerman V. I. Models of big data processing in massively parallel systems //Системы высокой доступности. – 2013. – Т. 9. – №. 1. – С. 035-043.

Lomet D. The evolution of effective b-tree: Page organization and techniques: A personal account //ACM SIGMOD Record. – 2001. – Т. 30. – №. 3. – С. 64-69.

Lvovich I. et al. Modeling and optimization of processing large data arrays in information systems //2021 International Conference on Information Technology and Nanotechnology (ITNT). – IEEE, 2021. – С. 1-5.

Monga, Vishal, and Brian L. Evans. "Perceptual image hashing via feature points: performance evaluation and tradeoffs." IEEE Transactions on Image Processing 15.11 (2006): 3452-3465.

Munerman V., Munerman D. Realization of Distributed Data Processing on the Basis of Container Technology //2019 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus). – IEEE, 2019. – С. 1740-1744.

Munerman V., Munerman D., Samoilova T. The Heuristic Algorithm For Symmetric Horizontal Data Distribution //2021 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus). – IEEE, 2021. – С. 2161-2165.

Munerman V.I. The experience of massive data processing in the cloud using windows azure (as an example) High availability systems. - 2014. - V. 10. - №. 2. - p. 8-13.

Pushpa Rani Suri, Sudesh Rani, "A New Classification for Architecture of Parallel Databases", Information Technology Journal, vol. 7, pp. 983. (2008).

Pyurova T. A., Skvortsov S. V. CUDA technology and parallel computing On GPU //Informatics and applied mathimatic: interuniversity compendium of treatises, no. 21, pp. 163-166. 2015. (In Russ).

Sridhar R. et al. Optimization of heterogeneous Bin packing using adaptive genetic algorithm //IOP Conference Series: Materials Science and Engineering. – IOP Publishing, 2017. – Т. 183. – №. 1. – С. 012026. 16.

Syrotkina O. et al. Mathematical Methods for optimizing Big Data Processing //2020 10th International Conference on Advanced Computer Information Technologies (ACIT). – IEEE, 2020. – С. 170-176.

Wajszczyk B., Gruszka I. M. Analysis of possibilities to increase the efficiency of the relative database management system using the methods of parallel processing //Radioelectronic Systems Conference 2019. – SPIE, 2020. – Т. 11442. – С. 385-398.

Zakharov V. et al. Architecture of Software-Hardware Complex for Searching Images in Database //2019 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus). – IEEE, 2019. – С. 1735-1739.

Zakharov V. N., Munerman V. I., Samoilova T. A. Parallel methods for deriving associative rules with the usage indatabase and in-memory technologies //CEUR Workshop Proceedings. – 2017. – С. 219-225.

Zobel J., Moffat A., Sacks-Davis R. An efficient indexing technique for full-text database systems //PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON VERY LARGE DATA BASES. – INSTITUTE OF ELECTRICAL & ELECTRONICS ENGINEERS (IEEE), 1992. – С. 352-352.


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность ИТ конгресс СНЭ

ISSN: 2307-8162