Tuesday, November 11, 2014

PHP Search Speed

I recently had a problem with a PHP command line script I use to sanity check some of our systems.

One particular check compares user accounts across a number of systems. There are ~100,000 user accounts. The script loads data in to memory from various LDAP and SQL interfaces. Once in memory, it iterates through the records and compares them. This process takes a few minutes, mostly spent loading data via LDAP.

I had identified a data problem in one system which uses SQL to get the user data. The SQL in question already joins a few tables and rather than introduce another complex join, I decided to just load the data about the new item into a new array using a separate SQL statement. This added just a few seconds to the script run time. However, once I implemented the record compare, the speed of the script slowed to a halt.

This was unexpected because I regularly process data sets this size and larger in PHP without and problems (once I raise the memory_limit).

This is what I discovered while debugging this problem:
It is well known that the PHP in_array & array_search functions are slow. They perform a linear search on the array resulting in O(n) performance.
Many people have contributed code which can implement a binary search on an array. This in theory reduces the performance to O(log n). My testing shows an improvement in search times but they are still quite slow and not representitive of the perforamce I am use to.
So how was it that I have never had this problem before? A quick inspection of my existing code shows that I always retrieve data by using the key. This results in a hashtable lookup of O(1).
So I changed my code to store my new data in the array key and use isset($array[$key]) and bang! my script is running on time again.

Footnote:

The performance of these functions may not be exactly O(n), O(log n) and O(1) but are close enough to enable comparing the performance of these functions against each other.

25 comments:

  1. That’s a great news, every country should take steps like this. geometry dash

    ReplyDelete
  2. I feel it interesting, your post gave me a new perspective! I have read many other articles about the same topic, but your article convinced me! I hope you continue to have high quality articles like this to share with everyone!
    happy wheels

    ReplyDelete
  3. The article you have shared here very awesome. I really like and appreciated your work. I read deeply your article, the points you have mentioned in this article are useful
    vex 3

    ReplyDelete
  4. Text API integration plays a vital role in enabling your existing applications or systems to be mobile. When you integrate your software or application with a reliable and feature-rich text API, then delivering real-time communication to your customers is easy.

    ReplyDelete

  5. لذلك تحرص شركتنا ارخص شركة مكافحة حشرات بالرياض كأفضل الشركات الموجودة فى الرياض على التخلص من جميع الأفات الشرسه مثلا الفئران وغيرها من القوارض التى من الممكن ان تكون سبب فى تدمير اغراض اى منزل او قد تسبب بعض الامراض اونقلها.شركة مكافحة النمل الاسود بالرياض _ شركة مكافحة الصراصير بالرياض _ شركة مكافحة البق بالرياض _ شركة مكافحة الوزغ بالرياض _ شركة مكافحة الثعابين بالرياض

    ReplyDelete
  6. What a nice Images are there in it helix jump

    ReplyDelete
  7. It is a great article. You will surely like this also because it is a great stuff

    Raft Wars 2

    ReplyDelete
  8. Really informative article.Really looking forward to read more. Really Cool. คาสิโนออนไลน์ที่น่าเชื่อถือและมีความเป็นมืออาชีพที่สุดในตอนนี้
    โปรโมชั่นGclub ของทางทีมงานตอนนี้แจกฟรีโบนัส 50%
    เพียงแค่คุณสมัคร สล็อตออนไลน์ กับทางทีมงานของเราเพียงเท่านั้น
    ร่วมมาเป็นส่วนหนึ่งกับเว็บไซต์คาสิโนออนไลน์ของเราได้เลยค่ะ
    สมัครสล็อตออนไลน์ >>> Goldenslot
    สนใจร่วมสนุกกับ คาสิโนออนไลน์ คลิ๊กได้เลย
    มีทั้งคาสิโนออนไลน์ หวยออนไลน์ ฟุตบอลออนไลน์ สล็อตออนไลน์ และอื่นๆอีกมากมาย

    ReplyDelete

  9. Welcome to the official website of our company to clean apartments, houses, villas and shops in Saudi Arabia This is because we are the best home services company at the moment and by the testimony of our valued customers, the workers who work in the company have a very high experience in the field of cleaning in general, as well as we will give you cleanliness of the highest quality and the lowest prices with the testimony of customers, our goal is to satisfy our customers to the highest degree possible and thanks to God always First of all, thanks to the workers and technicians, we were able to satisfy all the customers who contracted with the company and contacted them, because we specialize in cleaning, sewerage and transport of luggage just contact us and you will notice that your house as if it is completely new from the intensity of cleanliness.
    شركة الصفرات للتنظيف بالرياض
    شركة الصفرات لتنظيف الخزانات بالرياض
    شركة الصفرات لمكافحة الحشرات بالرياض
    شركة الصفرات لنقل الاثاث بالرياض
    شركة الصفرات لكشف التسربات بالرياض
    شركة تنظيف بالمدينة المنورة
    شركة تنظيف بينبع
    https://www.cleanriyadh.com/

    ReplyDelete
  10. considered as شركة نقل عفش بالمدينة المنورة The company has a fleet of vehicles equipped to transport the baggage to any area inside or outside Saudi Arabia, at the lowest possible cost, without any losses at all, where the employment When you contact us, our representative will contact you wherever you are, to inspect your existing furniture, determine the appropriate transportation method and the number of workers required to do so, agree on the appropriate time for the customer to transport, and determine the method of payment. Pric you, all you have is the contact numbers شركة نقل اثاث بالمدينة المنورة We pray wherever you are
    To visit the site
    https://www.elkhalegelaraby.com/
    خدمات نقل عفش واثاث

    ReplyDelete

  11. The task of moving a catcher needs to rely on شركة نقل عفش بالمدينة المنورة It is specialized and has a long history in this field, helping it to carry out the task of transporting the baggage without causing any problems. Among the precautions taken by the company during transport of the luggage are the following: High-quality packaging materials to ensure that no problems are encountered during transport, the luggage is loaded into the car very carefully so as not to be subject to any problems during transportation. The luggage is re-connected to the place where it is transported and the packaging that is placed on the pallet is removed. During transportation. , All the transferred items are written to make sure that there is no defect in the piece, therefore you will ensure the transfer of the baggage without being exposed to any problems during transportation, شركة نقل اثاث بالمدينة المنورة .
    Is characterized by شركة تنظيف بالمدينة المنورة With the experience and efficiency in the task of cleaning, as it uses the best equipment and the latest in addition to many aspects that focus on them to ensure that the cleaning is done to the fullest, and the most important things that you find in the fast service company and nowhere else find the following: It depends on the best detergents that can eliminate stains and smears, follow a certain system in the cleaning task; for this reason it ensures that it reaches the best results, depends on the latest methods such as steam cleaning method which proved its efficiency in the cleaning work, During which to eliminate all germs and The bacteria that are present in the place and maintain the health of the present, possess the best specialized perfumes that you can only find شركة نظافة بالمدينة المنورة.
    that شركة تخزين اثاث بالمدينة المنورة It can store any quantity of furniture even if it is huge. This is because the company owns large warehouses divided into many sections. It packs and puts furniture in cars and is transported to the stores equipped with the highest level and has the latest means of monitoring. شركة تخزين عفش بالمدينة المنورة You will get satisfactory results without having to pay a lot of money in return
    Be aware شركة تنظيف بينبع Cleaning works require the latest equipment for cleaning and the best detergents. This is what it provides to its customers. Therefore, it ensures the results. The company can clean any place, Will t D have شركة نظافة بينبع Discounts and offers can not be found elsewhere
    The task of moving furniture from one place to another can not be done by anything, but it must rely on a specialized company such as شركة نقل اثاث بينبع ، As it will provide you with assurance that the furniture transfer works will be done well and will not be exposed any piece of the pieces to any problems during the transfer of furniture. In addition, it will carry out a comprehensive inspection of all parts before and before the transfer. The best packaging tools are used in this task. you only have to communicate with شركة نقل عفش بينبع .
    Visit our website
    https://www.zeenclean.com/
    خدمات تنظيف

    ReplyDelete

  12. شركة تنظيف بالمدينة المنورة Is one of the best companies currently located in Al Madinah Al Munawwarah, Saudi Arabia. Since the company started its activities more than twenty years ago, it has endeavored to develop all its services and made sure to provide the best service at the highest level of quality in cleaning and disinfection at the lowest prices possible. The company believes that the company's employment is a mirror that reflects the efficiency of the company that boasts of it. The worker must be confident and credible to provide the customer with everything he can provide. JAL cleaning and sterilization found in شركة تنظيف منازل بالمدينة المنورة .
    شركة نقل اثاث بالمدينة المنورة Providing customers with transportation services of furniture and furniture and storage of furniture using the latest methods and methods and modern techniques and trained workers and freight cars specially equipped to transport furniture and clothes to anywhere inside or outside Medina, customers often need a city to transfer furniture to move furniture from their offices or homes Or their companies to new places but are afraid of their loss of damage or loss, but our company guarantees its customers to complete the transfer process safely and safely without being exposed to any piece of fracture or loss as the company provides a service to lift and download furniture using a hydraulic winch The only available within شركة نقل عفش بالمدينة المنورة .
    شركة تنظيف بينبع The company's services include all kinds of services needed by customers such as house cleaning services, apartments, cleaning services for ground and upper water tanks, cleaning of palaces and cleaning services for hotels, residential complexes, mosques, parks, facilities and others. Types of Services The main governmental and private bodies cooperated with the cleaning company in Yanbu. Our company has provided them with the most complete image that has impressed them and increased their confidence in the skill and efficiency of our company and its employees. P safe and effective when providing various cleaning services such detergents, but inside there is no شركة تنظيف منازل بينبع .
    شركة نقل اثاث بينبع Is one of the largest furniture transport companies in the city of Yanbu, which is known for providing services at the highest level and the company always offers offers and discounts and wonderful offers on the cost of services and this is what surprises customers and make them always put them the only choice for them, we have a professional team of carpenters who oversee the distinguished operations Dismantling and installing furniture and repair and maintenance of any fractures, as the company has workers polishing furniture and wooden furniture with effective polishes to remove the dirt from the layers of paint and make the furniture bright shiny like new, after the completion of cleaning the workers of evil Of furniture with materials encapsulates the strength and durability to protect the furniture from the external shocks or natural factors such materials are available only from within شركة نقل عفش بينبع
    Visit our website
    https://fajr-sate3.com/

    ReplyDelete

  13. شركة تنظيف بالمدينة المنورة Is one of the best companies currently located in Al Madinah Al Munawwarah, Saudi Arabia. Since the company started its activities more than twenty years ago, it has endeavored to develop all its services and made sure to provide the best service at the highest level of quality in cleaning and disinfection at the lowest prices possible. The company believes that the company's employment is a mirror that reflects the efficiency of the company that boasts of it. The worker must be confident and credible to provide the customer with everything he can provide. JAL cleaning and sterilization found in شركة تنظيف منازل بالمدينة المنورة .
    شركة نقل اثاث بالمدينة المنورة Providing customers with transportation services of furniture and furniture and storage of furniture using the latest methods and methods and modern techniques and trained workers and freight cars specially equipped to transport furniture and clothes to anywhere inside or outside Medina, customers often need a city to transfer furniture to move furniture from their offices or homes Or their companies to new places but are afraid of their loss of damage or loss, but our company guarantees its customers to complete the transfer process safely and safely without being exposed to any piece of fracture or loss as the company provides a service to lift and download furniture using a hydraulic winch The only available within شركة نقل عفش بالمدينة المنورة .
    شركة تنظيف بينبع The company's services include all kinds of services needed by customers such as house cleaning services, apartments, cleaning services for ground and upper water tanks, cleaning of palaces and cleaning services for hotels, residential complexes, mosques, parks, facilities and others. Types of Services The main governmental and private bodies cooperated with the cleaning company in Yanbu. Our company has provided them with the most complete image that has impressed them and increased their confidence in the skill and efficiency of our company and its employees. P safe and effective when providing various cleaning services such detergents, but inside there is no شركة تنظيف منازل بينبع .
    شركة نقل اثاث بينبع Is one of the largest furniture transport companies in the city of Yanbu, which is known for providing services at the highest level and the company always offers offers and discounts and wonderful offers on the cost of services and this is what surprises customers and make them always put them the only choice for them, we have a professional team of carpenters who oversee the distinguished operations Dismantling and installing furniture and repair and maintenance of any fractures, as the company has workers polishing furniture and wooden furniture with effective polishes to remove the dirt from the layers of paint and make the furniture bright shiny like new, after the completion of cleaning the workers of evil Of furniture with materials encapsulates the strength and durability to protect the furniture from the external shocks or natural factors such materials are available only from within شركة نقل عفش بينبع
    Visit our website
    https://fajr-sate3.com/

    ReplyDelete
  14. Thanks for the thoughts you are giving on this website. download clash of clans

    ReplyDelete
  15. Daftar situs judi online terpercaya dan terbaik di indonesia gabung sekarang....
    luxury138
    luxury138
    qq188
    qq188
    qq288
    qq288
    Pokerace99
    poker88
    Slot online
    betfortuna.

    ReplyDelete
  16. Pakarbet Merupakan Bandar Slot Online Terbesar dan Agen Judi bola88 Terpercaya Di indonesia yang menyediakan berbagai Permainan Judi Online Seperti Sportsbook, Live Casino, Slot Games, Poker Online dalam 1 userid bisa semua Games Dengan Promo dan Bonus Terbesa

    ReplyDelete
  17. Ayo Gabung di situs judi online Terpercaya dengan 1 userid bisa bermain semua games dengan bonus terbesar hanya di pakarbet

    bolapelangi
    bolapelangi
    indobola88
    indobola88
    itubola
    itubola
    afapoker
    afapoker
    bolapelangi

    ReplyDelete
  18. Anda Pencinta Games online ..?
    Kunjungi situs kami pakarbet

    1 Akun Bisa Bermain Sportsbook, Live Casino, Slot Online, poker Online, Ada Juga Live Roulette dan Sicbo Terbaik

    ReplyDelete
  19. This is the best article for reading and gets important information, reading the article is my hobby, but I really like read your article, amazing article thank you very much, allassignmenthelp is a wonderful platform to write your easy and article by the best expert. Pay For Assignment Help

    ReplyDelete
  20. Anda Pencinta Games online ..?
    Kunjungi situs kami pakarbet

    1 Akun Bisa Bermain Sportsbook, Live Casino, Slot Online, poker Online, Ada Juga Live Roulette dan Sicbo Terbaik
    Menang berapa saja di bayar...
    Whatsapp : +85585824263

    ReplyDelete