How we test smart home devices

PCMag editors independently select and review products. If you buy through affiliate links, we may earn commissions that help with our testing.

Google Nest Center — Source: Angela Moscaritolo

At PCMag, we test every type of smart home device on the market, including door locks, light bulbs, microwaves, plugs and power strips, security cameras, video doorbells, and more. We install these devices in our homes, where we conduct hands-on tests and compare them with similarly priced alternatives. We evaluate each product based on its features, interoperability with other smart products and standards, performance and price.

If you’re planning to equip your home with the latest smart technology, this is what we look for in each product category. We also have separate pages for testing robot vacuum cleaners and smart displays.

Devices

We review a wide variety of smart devices, including air conditioners, air purifiers, and kitchen gadgets.

In the case of air conditioners, we assess the complexity of the installation process and how to connect the power supply to the phone. We also test scheduling and preset options, voice commands, and interoperability with other smart devices. Most importantly, we count the time needed for the air conditioner to cool the room.

For air purifiers, we evaluate the installation and setup process and consider available features, such as the number of air quality measurements offered and the volume of each device. We also calculate how long it takes the purifier to clean the room of pollutants.

Kitchen gadgets and smart grills typically require extensive hands-on testing, which may include cooking meals, using pre-made one-touch cooking methods, loading recipes from a database, and using a mobile app and voice commands to prepare a meal from start to finish. It probably goes without saying that this is one of the most enjoyable product categories to test.

These aren’t the only devices we test, but many of the things we look for remain the same. We provide detailed comparisons and testing details in each of our reviews, so you’ll never wonder which product is right for your home.

Light bulbs

Testing smart bulbs involves assessing the installation process, light quality, connectivity options, integration, and companion app controls. We start by downloading the mobile app, adding a bulb or light strip to the app’s device list and giving it a name. If we are testing a lighting kit, we install all the bulbs in the package and assign them to a group.

For colored bulbs, we test any pre-programmed color settings and create custom ones to see how the bulb performs and evaluate color quality. We do the same with white light bulbs to evaluate the warm and cool temperature settings. We run some mobile app tests to see how responsive the bulb is (how quickly it responds to commands), and we test voice commands to see if the bulb supports Alexa, Google, or Siri. We also run routines or scenes to see how the bulb interacts with other smart devices and test its ability to keep to usage schedules.

Castles

Testing smart locks can be a bit more complex than testing most other smart home devices because we usually have to remove the existing lock assembly. Here, we’re looking for well-documented installation instructions and thoughtfully designed mobile apps that guide you through the entire process.

We test the accuracy of their fingerprint scanners and the responsiveness of their touch panels, as well as features such as voice control, access schedules, auto-lock options and geofencing. Additionally, we run Alexa and/or Google routines to see how the lock works with other smart devices such as cameras and lights.

Plugs and power strips

To test smart plugins, we start by downloading the companion app and creating a home account. We then evaluate the simplicity of the installation process. Most plugs only require you to insert them into a wall socket. You can then add them to the app by pressing the pairing button or connecting the device to your home network.

We test how the plugin responds to commands from the mobile app and the manual control button, as well as its ability to work with supported voice commands. If the plug works with other smart devices such as door locks, doorbells and security cameras, we configure the other device to trigger it through a routine or scene and evaluate how smoothly the process works.

We also create schedules to see how well the plugin follows them. If it reports power usage, we compare the plug’s energy readings to those of a competing plug or power meter.

Security cameras

To test indoor and outdoor security cameras, we install them according to the printed or in-app instructions. We take into account the difficulty of the installation, whether it requires cabling, the availability of mounting options and the length of the installation. When assessing a camera’s features, we consider its design (including IP rating) and video resolution, as well as features such as two-way audio, Wi-Fi connectivity, color night vision, motion and sound detection, and smart alerts. that distinguish between human, car and animal movements. We also note how cameras store motion-triggered footage (locally, in the cloud, or both) and whether storage and advanced features cost extra.

When assessing image quality, we pay attention to pincushion or barrel distortion, quality of night vision lighting, image sharpness and color saturation. Finally, if the camera supports Alexa or Google voice commands or works with Apple HomeKit, we test its ability to stream to a display hub and integrate with other connected smart devices such as locks, lights and plugs.

Security systems

We test DIY home security systems that you install and manage yourself, as well as those that require professional installation. In addition to price and features, we base our ratings on our DIY systems on the difficulty of installing individual components, such as door/window sensors and motion sensors. We also consider factors such as component size, response time (the time it takes the sensor to send an alert after a trigger event), and the price and availability of additional components. Additionally, we take into account the price of professional monitoring, any contractual requirements, and any additional features that unlock monitoring options, including a companion mobile app or free recording storage.

For professionally installed systems, we evaluate the ordering process and response time. We pay attention to the installation time taken by the technician, his professionalism and knowledge of the product. Other factors include the availability of battery and cellular power options, any contractual obligations (including early termination fees) and, of course, price.

Video intercoms

When testing video intercoms, we start by assessing the ease of installation of the device, especially if it requires work with a low-voltage electrical installation. If the bell is powered by a battery, we pay attention to how long the battery is expected to last between charges and the difficulty of the charging process.

We put video intercoms through the same image quality, motion detection and two-way talk tests that we use for indoor and outdoor security cameras. In this category, we’re looking for features like head-to-toe aspect ratio, third-party device integrations, and voice control support. We are also evaluating the mobile app and considering video storage options, as well as any applicable subscription fees.

The best smart home devices for every room

Overwhelmed by choices? Our interactive guide to the best smart devices will help you choose the right products for every room in your home. And if you live to rent, check out our suggestions for the best products that do not affect your deposit and that you can pack and take with you after the lease expires.