People who complain about conceptual art always do so on the grounds of craft. Anything that has no painterly or sculptural skill is not art, because anyone could do it.
this also absolutely holds true for experimental music or video art, and in fact it holds true largely across the spectrum of electronic music. i for one have gotten into the "that's not art" or "djs aren't real musicians" arguments so many times that i now pretty much refuse to participate. maybe that's selfish and arrogant of me, and i should be sharing my wisdom with those who don't know any better, no matter how aggravating it gets or how sick i get at hearing the same ignorant comments time and time again. but then again, the people i see most often espousing such "that's not music" arguments are actually trained musicians themselves... people who play "real" physical instruments, and quite likely have spent years getting "classical" training. music majors especially often tend to have the most conservative view of music imaginable, and anyone who hasn't put in the number of years of intensive study that they have get frowned upon as something less than "serious" musicians. i suspect a lot of this is jealousy: "why should they be considered musicians when i have to spend two hours a day practicing just to keep up?"
this is not to say that only trained musicians make those arguments. but they're the ones i see making them the most often. perhaps this is simply because the forums where i hang out tend to be music- or art-related forums full of musicians. but let's move on.
But when people object to individual pieces, it's almost always because of the subject matter. This has been true since the start of the readymade tradition - Duchamp, after all, could have made his point with any object, since his point was "this is art because I call it art". He could have used a foot bath, or scales, or even something that didn't belong in the bathroom at all. He used the urinal, presumably out of a sense of mischief, extra to his original purpose, but it's the mischief rather than the message which stuck. If he had used a washing-up bowl instead, I doubt he would be the father of conceptual art. I think someone else would have come along with a giant suppository.
The more tacit conservative requirement of art is that, besides being integrally skilful, it is also lofty in its subject matter (I don't mean "like a loft" - that would be only one step up from a shed). Heedless promiscuity, dead sharks and lights going on and off are not lofty; natural beauty, anything vaguely devotional, large tableaux of human endeavours such as wars - now there's lofty.
while this is certainly true in the visual art world, thankfully, i don't think this part holds true in the musical realm. pop music (and perhaps also folk music, parody music, etc) has acclimated the public to accept that music can be about pretty much anything, from "the lumberjack song" to "undone (the sweater song)" to "the thong song". people complain about the content of music all the time: they complain about explicit content, homophobia, misogyny, material that (they claim) is unfit for children, or even content that is too syruppy sweet to listen to without gagging. the most extreme complainers even want to ban music about certain subjects. but they never complain that "that's not music" because of the content.
You'd never hear people discussing the Booker Prize in these terms - it's unthinkable that a book would be criticised for a "nasty" plot or a load of swearing (the verbal equivalent of dung). It's accepted that just to ascend to the point of being published and receiving attention, an author must have invested enough seriousness in his endeavour that any topic will, ultimately, shed some light on the human condition, even if that topic is as childish as a talking tiger or as unseemly as paedophilia.
People assume this, often without reading the books, just as they assume the opposite about the Turner Prize, often without having seen the shortlisted artists' work. And for no better reason than that most of us had a verbal education up to at least 18, and a visual education that stopped at around seven. We trust authors not to gull us only because we trust ourselves to be able to tell if they're trying to.
the last non-elective visual art class i remember taking was in 7th grade. the last non-elective music class was probably even earlier. though this article is from a british paper, the same goes for the US. arts education is a joke. the only way to be exposed to experimental music (or conceptual visual art) is to seek it out. i actually did take a class in "electronics and computers in music" when i was an undergrad, but that was... you guessed it... elective. it was a great class, though; we got to study & listen to stuff like john cage and morton subotnick.
still, the average person has no such education in experimental music and thus has no frame of reference.¶