Sounding Out Car Noises and AI Autonomous Cars

Sounding Out Car Noises and AI Autonomous Cars 1By Lance Eliot, the AI Trends Insider

[Ed. Note: For reader’s interested in Dr. Eliot’s ongoing business analyses about the advent of self-driving cars, see his online Forbes column: https://forbes.com/sites/lanceeliot/]

The sounds of silence.

You might have thought that I was referring to the famous song by Simon and Garfunkel, but I am actually referring to the moments in which silence is golden. In particular, I’m referring to cars that are “silent” in that they aren’t making any rattling, grinding, clanging, or banging sounds as you happen to be riding in them.

Car Noises Usually Are A Bad Sign

Usually, any such ear catching sounds are bound to mean that there is something wrong with the car and you are likely at risk while riding in the vehicle.

At any moment, the engine might seize up, the axle might crack and fail, or any number of maladies might emerge.

The sounds emanating from the car can be a handy sign that something is amiss and often precedes the whole-hog failure of a part or element of the car.

Of course, sometimes the disturbing noise coincides with the actual failure event itself, in which case, the car problem is likely more apparent than the noise that you might have heard (the car sharply swerving, or the engine halting tends to draw more attention than the noise itself).

In college, I knew an amateur car mechanic that delighted in telling me what each car noise might mean in terms of the potential problem with a car.

He was a walking and talking encyclopedia of car sounds and noises. He had an audible superpower worthy of being featured in a Marvel movie or comic book series. You could ask him what it means to hear a clanking, followed by a clicking, followed again by more clanking. He could tell you that it was likely you had a problem with your suspension, or the brakes, or the exhaust system, or whatever part of the car that he figured made those kinds of noises.

There’s a popular radio and TV commercial airing recently that has people making seemingly silly noises with their mouths and noses, attempting to replicate the sounds they heard their car making. A savvy car specialist that has “heard it all” watches them make these noises and calmly tells them what is most likely wrong with their cars. I’d bet that you at one time or another have tried to describe an issue of your car by telling others that the car was making a weird or unusual noise, and you valiantly tried to make the sound by vocalizing it. We all do so.

As an aside, my writing down these sounds herein and using words such as bam, bang, clang, are considered examples of onomatopoeia.

When you say that a cow goes moo or a sheep goes baa-baa, you are making use of an onomatopoeic word. These are words that attempt to reflect phonetically the sound of something and somewhat resemble or suggest the actual sound that we hear. If you didn’t already know that word, figured you’d want to add it to your vocabulary.

A car does not necessarily need to make any special sounds or noises before something goes awry.

I mention this aspect because you can have a part or element that decides to quit on you and can do so without having to make an adverse and noticeable sound. I had a timing belt that just abruptly snapped on my car. There usually is a warning, often heard as a slipping sound of the belt not being able to remain clinging in place. You’ve undoubtedly heard a slipping belt sound many times. In this somewhat rare instance, my car belt opted to completely snap in two and did so without being polite enough to warn me beforehand.

Speaking of being polite, in a manner of speaking, the untoward sounds and noises that a car makes is actually a handy and helpful act for us humans. Assuming that the sound or noise occurs before a bad turn of events occurs, the car is presumably trying to give you a heads-up to be on the watch. People that choose to ignore these warning sounds are doing so at their own peril.

I had a friend that bitterly complained that his car kept making numerous horrible noises.

Darn it, he proclaimed, he wished the darn-blasted car would stop making all those irritating and excruciating sounds. I would ask him whether he thought that the car should magically self-repair itself and thus extinguish making these noises?

I would further point out that the noises are a welcome reminder that maybe he ought to have someone look at his car and figure out what is wrong with it. Perhaps now might be timely to make needed repairs or perform car maintenance that he had most likely forsaken to do. Imagine if he opted to drive on a long trip in a barren place, let’s say driving through a desert, and the car falls apart, which he could have anticipated beforehand by the noises the car was making and hopefully have not gotten stuck in the middle of nowhere.

Car Noises Are A Helpful Sign

For me, I’m appreciative when a car makes those seemingly unwelcome sounds or noises.

How lucky that the car has kindly chosen to let me know something is amiss.

If you had a pet dog or cat, and the pet started to make untoward sounds, you’d likely be worried that something was amiss. You’d look closely at the dog or cat, trying to figure out what is wrong. Assuming you could not readily discern the problem, the odds are that you’d take your pet to see a veterinarian. I suppose you might first tell your friends the sounds your pet was making, in hopes they might know what the problem is, doing so before having to go to the trouble to visit a veterinarian.

We tend to do the same thing with our cars.

There’s an untoward sound or noise. You might tell a friend about it, asking them if they’ve ever heard such a noise. The friend might say they had the same sound when the car needed a tune-up, and so you opt to take your car into a tune-up shop. Or, your friends might say they’ve never heard such a sound in a car before. You then need to decide whether to take the car to a repair or maintenance facility.

Admittedly, none of us want to hassle taking our car in.

You need to go out of your way to drive to such a place. You need to wait for a trained mechanic or similar to take a look at your car. You need to have them diagnose what it might mean. Then, you fearfully wait to hear how much it will cost to repair it and dread the length of time needed for your car to be in the shop and unavailable to you. The whole process is daunting and undesirable, for sure.

Your mind considers other possibilities. Maybe the sound isn’t so bad. It could be a sound that will go away on its own. If you ignore it, somehow the car will realize that you aren’t giving in to the irritating sound. It’s like a child that keeps pestering you to take them to the ice cream parlor. Let the kid ask a bunch of times, ignore the requests, and in the end perhaps the child will give up asking about it. Could car sounds be that way too, you wonder?

The thing is that sometimes a car sound does indeed go away.

This emboldens us to consider ignoring such sounds or otherwise discounting them. I knew it, you might say to yourself, I successfully ignored that sound for two weeks and now it has stopped making the irksome noise. Thank goodness you didn’t take the car into a mechanic. All they would have done was charge you a fee to tell you that the noise was nonsense and you can merely continue to drive the car as usual.

This idea that the sound has disappeared and there is no concern that the sound occurred to begin with, well, that can also be a kind of lazy person’s trap.

It could be that the thing that is going wrong has gotten even worse. In the worsened state, the noise might no longer be generated. I know that seems counter-intuitive, namely that if the problem is getting worse then the noise certainly should be getting worse, but not everything works that way. You can have a problem that in fact is worsening and yet the sound either goes away, or it becomes some other kind of sound, perhaps a sound less obvious or less threatening.

Car Sounds That Come And Go

The other perturbing aspect is that suppose you do opt to take your car into a mechanic, and yet the sound perchance no longer is being emitted.

The car mechanic might look at you as though you are crazy. I swear on my solemn oath, you tell the mechanic, it was making this clang-clang sound. The mechanic starts the car, listens carefully, listens patiently, but no such sound occurs. Shrugging their shoulders, the mechanic might say come back once it actually is making the sound and they will take another look.

Equally frustrating involves the mechanic not only unable to hear the sound, and an assertion that without the sound they cannot tell you what might be wrong, but they then offer they must do a complete diagnosis of your car. Big bucks! This is likely to have you roll your eyes in despair and disgust.

Rather than having the mechanic focus on the specific aspect that is presumably wrong with the car, the mechanic is going to charge you to go on a fishing expedition. You know in your bones that the mechanic is going to find 20 things wrong with your car, even though that’s not what you came into the mechanic for. You went from one little itty bitty sound to instead having to get your entire car nearly overhauled.

On the topic of trying to use the car’s sounds for diagnosis purposes, it at times makes sense that the mechanic might not hear the noises that you heard. Suppose the sound only arises when the car is actually in-motion. If you take your car to the repair shop and while it is parked you turn on the car, this might not be a sufficient replication of the circumstances under which the sound is generated. There is a strong possibility that the noise only arises when the car is in actual motion.

I knew one person that drove their car around and within the parking lot of the repair shop and still could not get the noise to appear. This is another viable possibility since the noise might only arise when the car is in faster motion, perhaps while going 65 miles per hour on the freeway. Going a few measly miles per hour in a parking lot at a snail’s pace could be insufficient to get the sound to announce itself.

Another factor can involve whether the car is heated-up.

Sometimes a noise might occur only when the engine is cold and upon first starting the car. In other cases, the noise only arises after the car has been going for a solid 30 minutes or more and the engine and the car components are heated-up. This undermines those occasions whereby you take your car to the repair shop, let it sit there, parked and the engine is no longer going, and they then turn it on and try to hear the sound that you experienced earlier.

The number of variants is rather daunting.

Maybe the sound only occurs while shifting gears. Perhaps the sound only happens when the car is taking a sharp turn. The noise might arise when going uphill, or only when going downhill. This highlights that where the car is being driven can make a difference in stoking the noise, along with how the car is being driven.

I knew someone that used to ride the brakes all of the time. At one point, she complained that whenever she put her foot on the brakes, there was a loud shrieking sound. I took the car for a short drive to see if I could hear the sound. Upon doing so, I was unable to replicate the noise. I then tried to put myself into her mindset, or you might say into her shoes, and I decided to start riding the brakes continuously. I would normally only use the brakes as needed and when needed. In any case, sure enough, after a few minutes of continually riding the brakes, the shrieking sound arose.

When I got back after doing the test drive, I was actually tempted to say that the shrieking sound was a child’s squeaky toy that was jammed under the brake pedal.

Ha!

Actually, though I make light of the situation, there are car mechanics that I’ve spoken with that claim they have had similar kinds of circumstances. They popped up the hood and found a kid’s toy that somehow got stuck into part of the car. One joked that after a complaint of banging sounds coming from the trunk area, the mechanic “discovered” that it was Jimmy Hoffa, trying to knock-knock and get out of the trunk (I told the mechanic that Jimmy Hoffa jokes are wearing thin as the Gen Z isn’t quite in tune with such humor).

A key aspect about car sounds is that you usually need to know when they arose.

Where was the person driving the car?

Type of road surfaces can make a difference, elevation can make a difference, and other roadway and infrastructure factors come to play.

How were they driving, such as with a lead foot or a light touch?

At one point of a driving journey did the sound begin, at the start or midway or towards the end?

Yet another factor is whether the sound is a one-time instance, or whether it is intermittent or periodic. One-time only sounds can be especially difficult to figure out. With the intermittent or periodic sounds, you usually have a greater chance of hearing it and also speculating as to what might be causing it.

There is the sound that is subtle and barely audible, while there are the noises or sounds that are unmistakable and boldly proclaimed. A sound might begin with barely being heard and gradually become an overwhelming grinding or pinging noise. On the other hand, the sound might begin with a huge and shocking bang, and then become a low hum that you are unsure whether it is making any sound or not.

I’ve also not yet mentioned that the sound or noise might change over time and get intertwined with other sounds too.

It could be that the first sound you hear is a hissing. The hiss seems to disappear. You then hear a rumbling noise. The rumbling noise dissipates. Now you hear an ugly popping sound. What is going on? Is your car possessed by evil spirits?

You continue driving and keep your fingers crossed that these noises don’t mean that your car is about to fall apart while on the road. You begin to hear the hissing sound again. Oh no, it has returned. This time though you hear the hiss and then you begin to also hear the rumbling noise.

Yikes!

Then, the popping sound joins into the cacophony. You are having your own car musical band that has decided to play all the instruments at once. It’s a scary rendition.

Here’s another scenario. I was in someone else’s car and thought I heard a rattling sound.

The other person was driving, and we were zooming along on the German autobahn. I looked furtively at the driver and was trying to see whether she had heard the sound. She seemed not to have heard it. The radio was on and there was music playing through the impressive speakers of the car.

I realized that the music was potentially masking or overpowering the rattling sound that I had heard. I reached over and dialed down the volume of the radio. Sure enough, the driver then heard the sound. I guess my ears were a bit more tuned to hearing car sounds. Or, perhaps the music that was playing on the radio wasn’t to my liking and I wasn’t listening intently to it.

The point being that there might be other sounds or noises occurring inside the car and might have nothing to do with car-specific sounds or noises. This can make it hard to differentiate the car-specific sounds from the rest of the sounds occurring in the car. My ears were able to differentiate the music from the rattling sound. Not everyone would necessarily be able to separate out such sounds. Realizing this, I had purposely turned down the volume so that the driver could also hear the sound.

Patterns And Car Diagnosis Via Noises

You might be thinking that these various car sounds and noises that relate to something untoward about the car are all ad hoc and have no pattern to them.

Not so!

As mentioned, car mechanics can often make a pretty good guess about what might be wrong with your car via hearing the sound that it is making.

The car mechanic doesn’t necessarily need to hear the actual sound and can use your description of the sound to make a guess about what is wrong. Your description of the sound might be in words only (possibly using onomatopoeic words), or you might try to replicate the sound by making your own gurgling and popping noises.

Today, there are several online databases of car sounds and noises, along with indicating what the noise or sound might suggest about any problems of your car. You can go to the database and listen to the sounds, trying to see if any match to the sound you hear in your car. You can also try recording the sound that occurs in your car and try to get it to match to the database, doing a calculation-type of matching using a sound conversion algorithm.

I don’t want to overstate the use of such online databases. Your particular brand and model of car might make a difference in the kind of sound emitted from your car and not match well to what’s in the database. The sounds in the databases are at times somewhat generic, or in other cases are specific to particular car brands and models. I’d suggest that though these repositories are handy, but you are still going to ultimately need an actual car mechanic to take a look and listen to your car.

I’d also say that those of you that might decide to record a sound and then when you get home try to look it up, keep in mind that this might be a chancy act of delay. Suppose the sound is one that is serious enough that something bad is going to happen to your car right away. You might want to get that car to a repair shop, rather than waiting to try and figure out what the problem is.

For those of you that are versed enough to be your own car mechanic, I don’t want you to feel that I’m pushing everyone to go to a car mechanic. If you have the knowledge and experience to care for your car, that’s great! I am merely attempting to emphasize that sometimes people don’t take their car noises seriously and therefore get themselves into worse trouble. In whatever manner you react to your car sounds, make sure to give them their due, taking action as needed.

For my article about sounds and sensory illusions, see: https://www.aitrends.com/selfdrivingcars/yanny-vs-laurel-sensory-illusions-and-ai-self-driving-cars/

For my article about diagnosing car-related problems, see: https://www.aitrends.com/selfdrivingcars/debugging-of-ai-self-driving-cars/

For what happens when a car freezes up, see my article: https://www.aitrends.com/selfdrivingcars/freezing-robot-problem-and-ai-self-driving-cars/

For the difficulties of irreproducibility of problems, see my article: https://www.aitrends.com/selfdrivingcars/irreproducibility-and-ai-self-driving-cars/

Car Sounds Detection And AI Autonomous Cars

What does this have to do with AI self-driving driverless autonomous cars?

At the Cybernetic AI Self-Driving Car Institute, we are developing AI software for self-driving cars. Detecting car sounds and noises is a helpful and some might say essential added element that the AI ought to be providing in a self-driving car.

Allow me to elaborate.

I’d like to first clarify and introduce the notion that there are varying levels of AI self-driving cars. The topmost level is considered Level 5. A Level 5 self-driving car is one that is being driven by the AI and there is no human driver involved. For the design of Level 5 self-driving cars, the automakers are even removing the gas pedal, the brake pedal, and steering wheel, since those are contraptions used by human drivers. The Level 5 self-driving car is not being driven by a human and nor is there an expectation that a human driver will be present in the self-driving car. It’s all on the shoulders of the AI to drive the car.

For self-driving cars less than a Level 5, there must be a human driver present in the car. The human driver is currently considered the responsible party for the acts of the car. The AI and the human driver are co-sharing the driving task. In spite of this co-sharing, the human is supposed to remain fully immersed into the driving task and be ready at all times to perform the driving task. I’ve repeatedly warned about the dangers of this co-sharing arrangement and predicted it will produce many untoward results.

For my overall framework about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/framework-ai-self-driving-driverless-cars-big-picture/

For the levels of self-driving cars, see my article: https://aitrends.com/selfdrivingcars/richter-scale-levels-self-driving-cars/

For why AI Level 5 self-driving cars are like a moonshot, see my article: https://aitrends.com/selfdrivingcars/self-driving-car-mother-ai-projects-moonshot/

For the dangers of co-sharing the driving task, see my article: https://aitrends.com/selfdrivingcars/human-back-up-drivers-for-ai-self-driving-cars/

Let’s focus herein on the true Level 5 self-driving car. Much of the comments apply to the less than Level 5 self-driving cars too, but the fully autonomous AI self-driving car will receive the most attention in this discussion.

Here’s the usual steps involved in the AI driving task: 

  • Sensor data collection and interpretation 
  • Sensor fusion 
  • Virtual world model updating 
  • AI action planning 
  • Car controls command issuance

Another key aspect of AI self-driving cars is that they will be driving on our roadways in the midst of human driven cars too. There are some pundits of AI self-driving cars that continually refer to a utopian world in which there are only AI self-driving cars on public roads. Currently there are about 250+ million conventional cars in the United States alone, and those cars are not going to magically disappear or become true Level 5 AI self-driving cars overnight.

Indeed, the use of human driven cars will last for many years, likely many decades, and the advent of AI self-driving cars will occur while there are still human driven cars on the roads. This is a crucial point since this means that the AI of self-driving cars needs to be able to contend with not just other AI self-driving cars, but also contend with human driven cars. It is easy to envision a simplistic and rather unrealistic world in which all AI self-driving cars are politely interacting with each other and being civil about roadway interactions. That’s not what is going to be happening for the foreseeable future. AI self-driving cars and human driven cars will need to be able to cope with each other.

For my article about the grand convergence that has led us to this moment in time, see: https://aitrends.com/selfdrivingcars/grand-convergence-explains-rise-self-driving-cars/

See my article about the ethical dilemmas facing AI self-driving cars: https://aitrends.com/selfdrivingcars/ethically-ambiguous-self-driving-cars/

For potential regulations about AI self-driving cars, see my article: https://aitrends.com/selfdrivingcars/assessing-federal-regulations-self-driving-cars-house-bill-passed/

For my predictions about AI self-driving cars for the 2020s, 2030s, and 2040s, see my article: https://aitrends.com/selfdrivingcars/gen-z-and-the-fate-of-ai-self-driving-cars/

Detection Then Diagnosis

Returning to the topic of car sounds and noises, let’s consider how this might be handled by AI self-driving cars.

First, true Level 5 AI self-driving cars will be equipped with internal microphones for the primary purpose of interacting with human passengers.

Humans inside an AI self-driving car will most certainly need to have an ongoing dialogue with the AI system, using an AI-equipped Natural Language Processing (NLP) capability, so that the human can indicate where they want to go. In addition, the human and the AI will likely have discussions about intermediary potential destinations and cover other topics such as whether the ride should be scenic or focused on getting to the desired destination and similar matters.

With those microphones already inside an AI self-driving car, the potential for also using that equipment as listening devices to detect car sounds and noises is readily heightened. I realize you might assume that the microphones would absolutely be used to listen for car noises, but I’ll offer some caveats about why that might not be quite so certain.

One issue that has yet to be resolved involves the privacy elements of passengers inside true Level 5 AI self-driving cars.

While you are a passenger in an AI self-driving car, should the microphones be activated and possibly recording your every word? For some, this is a significant potential intrusion into their privacy. Though some automakers and tech firms might say they won’t keep the recorded conversations, this is not assured and there could be reasons that the automaker or tech firm might feel compelled to give over such recordings (due to law enforcement or court requests, etc.).

If you believe the existing makers of various speech-oriented devices such as Alexa and Siri, it is claimed that until those systems hear the prearranged designated word, such as the name of the service, there is nothing being recorded. Instead, the NLP is supposedly scanning streaming audio and looking only for the activation word. Once the activation word is heard, the NLP then starts processing the conversation and may or might not be keeping a recording of the audio.

For more about privacy and AI, see my article: https://www.aitrends.com/selfdrivingcars/privacy-ai-self-driving-cars/

For NLP aspects, see my article: https://www.aitrends.com/selfdrivingcars/car-voice-commands-nlp-self-driving-cars/

For my article about socio-behavioral interactions, see: https://www.aitrends.com/features/socio-behavioral-computing-for-ai-self-driving-cars/

For my article about the advent of ridesharing via AI self-driving cars, see: https://www.aitrends.com/selfdrivingcars/ridesharing-services-and-ai-self-driving-cars-notably-uber-in-or-uber-out/

Let’s then consider what might occur while you are inside an AI self-driving car.

If the AI is only listening for the magical keyword for activation, this implies it is not activated per se until it hears the keyword, and therefore it is not actively listening for car sounds or noises. In that case, the only time at which the AI has an opportunity to listen for car sounds and noises would be once the human passenger has uttered the activation word.

You might at first say that waiting until the passenger says the activation word seems fine. The problem of course is that if the sound occurs prior to the passenger saying the activation word, the AI won’t know or won’t have recorded the sound. This could mean that a crucial sound or noise was not detected. It could also be that once the passenger has uttered the activation word, perhaps the sound does not repeat itself. This is nearly akin to going to see your car mechanic and claiming you heard a sound, but the car mechanic has no means to hear it themselves.

This gap in hearing such sounds or noises is a trade-off of the potential privacy intrusion.

One also has to consider whether the passenger will even have the presence of mind to activate the AI if there is a noise or sound heard. It could be that the passenger fails to realize that they could have the AI help in trying to determine what the sound entails. Plus, if a child is in the self-driving car and there is no adult present, it would seem less likely that the child would realize it might be prudent to activate the AI in the matter.

You might suggest that the AI would be activated when a human utters a prearranged keyword, plus it would activate automatically whenever a car sound or noise is detected. In that case, the AI NLP is searching for not just a particular keyword. It is also searching for various kinds of sounds. Once one of those sounds or noises is detected, the AI would potentially start to more actively listen, and might alert the human passenger accordingly.

The downside with this approach is that there is a chance that other sounds or noises that are not especially related to the car itself might inadvertently cause this kind of activation. You can imagine the potential irritation to a passenger if they are playing some game or music inside the self-driving car and the AI keeps falsely alerting that it believes a car sound or noise has occurred. The potential for false positives might be so much that passengers would not want the AI to be activating when it hears such sounds.

Another aspect to be considered is whether the “normal” microphones used to interact with the passengers will be sufficient for purposes of identifying and “hearing” the car sounds and noises that might occur.

If the microphones are positioned in a manner to pick-up the human voice, they might not be well-positioned for picking up car sounds or noises, which would usually emanate from below the flooring of the car. This might necessitate adding additional microphones into the self-driving car, which also adds cost, weight, and potentially impacts the interior design and space available. It might even make sense to have specialized microphones that aren’t for conversing with humans and instead installed solely for detecting the car sounds and noises.

This does bring up another facet of the car sounds and noises. Suppose a human passenger hears a car sound or noise, and they wonder what it might mean. It is conceivable the human passenger might ask the AI about the sound or noise. Did you hear that, the human asks the AI. What was that sound, the humans asks the AI.

I know that some AI developers cringe at having to add such aspects to the NLP dialogue capabilities of the AI system. For most AI developers that are currently working on self-driving cars, they are already overwhelmed with other matters that they consider more important. To them, the detection of car sounds is a far cry from anything core to a self-driving car. They would say that this is all an “edge” problem, meaning something that is at the far edge or corner of what really needs to be done, and so can be set aside until a much later time to consider it.

For my article about edge problems in AI, see: https://www.aitrends.com/selfdrivingcars/edge-problems-core-true-self-driving-cars-achieving-last-mile/

For my article about AI developer burnout, see: https://www.aitrends.com/selfdrivingcars/developer-burnout-and-ai-self-driving-cars/

For the dangers of AI groupthink, see my article: https://www.aitrends.com/selfdrivingcars/groupthink-dilemmas-for-developing-ai-self-driving-cars/

For the Top 10 trends in AI self-driving cars, see my article: https://www.aitrends.com/selfdrivingcars/top-10-ai-trends-insider-predictions-about-ai-and-ai-self-driving-cars-for-2019/

Human Passenger Interaction With The AI System

In any case, it certainly seems likely that a human passenger would ask the AI about the car sounds or noises.

If you were in a ridesharing car or a cab driven by a human, and you heard a seemingly disturbing sound coming from the car, I would bet that you’d ask the human driver about it. You’d want to validate that they heard the sound too. You’d hope that the driver might know what the sound presages. You would want the driver to be on the alert about the driving of the car, based on qualms about hearing untoward sounds or noises emanating from the car.

Human passengers in an AI self-driving car are going to react in the same manner, beseeching the AI to verify having heard the sound, along with aiming to get some explanation from the AI about it. Most importantly, the human passenger wants the AI to be watchful of the self-driving car since the sounds might foretell something bad about to happen. Will the AI be able to handle the driving safely if the sound is a precursor to a physical breakdown of some part of the car?

This is where the AI and NLP aspects get quite tricky. Suppose the AI says that there is a 30% chance that the knocking sound means that a rod in the engine is about to fail. Yikes, the passenger screams! Get me out of this self-driving car, now! This estimation of the probability might falsely give the impression to the human that the whole car is about to fall apart. It could cause a needless panic.

This raises the important question about how the AI should interact with humans about the sounds or noises that were heard. Do you want an AI that offers soothing words? Do you want an AI that tells it like it is, though the indication might be perceived as brash or harsh? A human driver would likely alter their response to a human passenger depending upon a multitude of factors, such as their own perceived seriousness about the noise, and the nature of the passenger and how they might cope with such a discussion.

We also need to consider that a human passenger might get startled by a sound or noise that has nothing to do with the car at all. Suppose the self-driving car happens to drive past a construction site and there is an explosive sound that comes from the construction taking place there. The human hears the loud bang and thinks it came from the self-driving car. The human asks the AI what the sound was.

If the AI did not hear the sound, perhaps being muffled by the car windows being closed, it might say that it did not hear the sound. This could spark the person toward further panic because they might then believe that the AI is not paying attention or has failed to detect something that might be very vital to the self-driving car safety.

One potential advantage of having the AI detecting for car sounds and noises would be that it might be able to do an extensive analysis on the car sound. A human driver of a ridesharing service might not know much about cars. For them, any sounds are just sounds, and they might not have a clue as to what a particular ping or pop signifies. The AI could look-up the sounds in a database and potentially have a better means of discerning what the sound portends.

We could carry this capability to a more advanced level for the AI.

Suppose the AI system had a knowledge-based component that could do an in-depth diagnosis to a level similar to a car mechanic hearing a noise. This more advanced capability could aid the self-driving car in diagnosing the potential problem and would also aid the ongoing driving of the self-driving car by the AI.

For knowledge-based system and AI self-driving cars, see my co-authored article: https://www.aitrends.com/selfdrivingcars/expert-systems-ai-self-driving-cars-crucial-innovative-techniques/

For emotions and AI aspects, see my article: https://www.aitrends.com/selfdrivingcars/ai-emotional-intelligence-and-emotion-recognition-the-case-of-ai-self-driving-cars/

For my article about OTA, see: https://www.aitrends.com/selfdrivingcars/air-ota-updating-ai-self-driving-cars/

For fail-safe AI systems aspects, see my article: https://www.aitrends.com/selfdrivingcars/fail-safe-ai-and-self-driving-cars/

If the sound occurs whenever the AI takes a sharp left turn, the AI action planning portion might opt to be especially careful when making left turns. This could avoid for the moment further harming whatever mechanical issue has occurred.

Via OTA (Over-the-Air) electronic communications, which the AI usually uses to pump data up to the cloud of the automaker or tech firm, and gets patches and updates pushed down to the on-board AI system, the AI could share the sounds or noises into the cloud. This might be handy as there might be other similar noises or sounds that have been heard in other AI self-driving cars of this particular fleet.

If the AI self-driving car is part of a ridesharing service, the AI might then confer with some master scheduling system and indicate that the self-driving car needs to get over to a repair shop. The self-driving car might finish up the delivery of the human passenger and then route itself over to the maintenance facility. The cloud would meanwhile realize that this AI self-driving car is temporarily out of service.

AI Planning Action Steps

In terms of the types of actions that the AI might take, doing so after detecting and diagnosing a car sound or noise, these are potential AI planning action steps: 

  • Stop use of the self-driving car “immediately” and pull over safely 
  • Stop use of the self-driving car as soon as practical and seek to get to a repair shop 
  • Adjust the driving of the self-driving car based on the potential issue 
  • Continue normal use of the self-driving car and note to have the matter checked 
  • Consider the sound or noise irrelevant and take no other action due to it 
  • Etc.

Another potential recourse would be for the AI self-driving car to institute an electronic dialogue with other nearby AI self-driving cars.

Using V2V (vehicle-to-vehicle) electronic communications, the AI self-driving car that has detected an untoward sound might request that other nearby AI self-driving cars take a look at the AI self-driving car to see if they detect anything untoward. For example, if the tailpipe is dragging on the street, an AI self-driving car that is behind this AI self-driving car could detect the dragging tailpipe via the use of its cameras.

The V2V dialogue might also be used to warn other nearby AI self-driving cars that the AI self-driving car initiating the dialogue might soon be exhibiting driving issues. Perhaps the self-driving car is going to start weaving, due to a mechanical breakdown, and the AI will be doing whatever it can to minimize the weaving. Meanwhile, other nearby AI self-driving cars have intentionally given a wider berth for the weaving AI self-driving car, doing so as a result of being forewarned about a potential car problem.

In addition, the use of V2I (vehicle-to-infrastructure) electronic communication might be useful. The AI self-driving car that has encountered untoward sounds might warn the roadway infrastructure that the self-driving car might soon be having problems. Furthermore, it could be that the roadway has a pothole and the self-driving car got damaged by hitting the pothole. The V2I might mention that the pothole has caused a problem. This could then be relayed by the V2I infrastructure to other AI self-driving cars that come along that roadway. Plus, hopefully, it would spur the roadway repair crews to fix the pothole make sure that it can no longer damage passing cars.

Besides using a knowledge-based system and a database to try and diagnose the car sounds, another approach would be via the use of Machine Learning (ML) and Deep Learning (DL). If an entire fleet of AI self-driving cars are reporting various sounds and noises, along with the later-on determination by car mechanics of what it signified, this kind of data could be used for pattern matching.

Using a large-scale or “deep” artificial neural network, we could feed this data into the ML or DL and see if it can train on the data. By doing so, whenever an AI self-driving car perchance encounters such a sound or noise, the ML or DL could be used to more definitively diagnose it. This could either be an on-board ML or DL that is placed inside the AI self-driving car or might be a cloud-based version and the AI self-driving car would access it via OTA or the equivalent.

For my article about Deep Learning, see: https://www.aitrends.com/selfdrivingcars/plasticity-in-deep-learning-dynamic-adaptations-for-ai-self-driving-cars/

For federated Machine Learning, see my article: https://www.aitrends.com/selfdrivingcars/federated-machine-learning-for-ai-self-driving-cars/

For Machine Learning benchmarks, see my article: https://www.aitrends.com/selfdrivingcars/machine-learning-benchmarks-and-ai-self-driving-cars/

For my article about ensemble Machine Learning, see: https://www.aitrends.com/selfdrivingcars/ensemble-machine-learning-for-ai-self-driving-cars/

Conclusion

Bang. Hiss. Ping. Pop. Clunk. Clack. Buzz. Rumble. Screech.

If your car is making these sounds, and if it isn’t because you are playing some music or having a wild party inside the car, you ought to consider what your car is trying to secretly tell you. Might be a loose fan belt. Maybe your brakes are wearing out. The muffler on your car might be busted. Your bumper might be dragging on the ground.

AI self-driving cars will have lots of sensors that are pointing outward to detect what is happening around the AI self-driving car. I’ve repeatedly called for AI self-driving cars to be outfitted with more sensors for purposes of AI self-awareness. By this, I mean that the AI self-driving car needs to know its own internal status.

For my article about AI self-awareness, see: https://www.aitrends.com/selfdrivingcars/self-awareness-self-driving-cars-know-thyself/

For my article about LIDAR and other sensors, see: https://www.aitrends.com/selfdrivingcars/lidar-secret-sauce-self-driving-cars/

For the dangers of myopic sensors, see: https://www.aitrends.com/selfdrivingcars/cyclops-approach-ai-self-driving-cars-myopic/

For what happens when sensors fail, see my article: https://www.aitrends.com/selfdrivingcars/going-blind-sensors-fail-self-driving-cars/

One important way to determine the status of the AI self-driving car will be via the sounds and noises that the car makes. Humans use those sounds and find them useful as a premonition or forewarning that something might soon go horribly awry. We ought to expect the AI to do the same.

With the AI doing sound detection, there is a good chance of boosting how these sounds will be dealt with. The AI self-driving car can be tracking how often the sound occurs, along with the circumstances such as going uphill or downhill, making turns or going straight ahead, speeding up or slowing down, and so on. With an advanced knowledge-based system, and perhaps Machine Learning or Deep Learning, when also combined with OTA, V2V, and V2I, we can substantially boost the diagnostic capabilities for AI self-driving cars.

It would also be hoped that this will further translate into safer driving by the AI system since it will be using essential clues in determining how best to safety drive the self-driving car.

That’s music to my ears, or, I might say are the sounds that I want to hear.

Copyright 2019 Dr. Lance Eliot

This content is originally posted on AI Trends.

Sounding Out Car Noises and AI Autonomous Cars 2

Read more

%d bloggers like this: