To start with, here are some things that Joyce says in chapter 2 (I read the preface and chapter 1 long enough ago that I'll have to review before saying anything about them) that strike me as being questionable or else just wrong. I'm not talking about these as an exercise in shooting fish in a barrel (if you think Joyce is obviously very wrong) or big game hunting (if you think he's much closer to the truth than that), but as a first attempt to case the joint or sniff out potential weak points in an argumentative structure with some promising foundations (e.g. the influence of Philippa Foot, who was influenced in turn by Anscombe).
On p. 31 Joyce announces that the task of chapter 2 is to clarify the inadequate response "Because you mustn't" to the question "Why musn't I do what's wrong?" The problem, it seems to me, begins when the child we are asked to imagine is told not to pinch another child "because it's wrong." (Of course it might really begin when the child wants to know why he can't pinch another child, or when he gets the urge to pinch, or when human nature became sadistic in this way, but I'll leave that aside for now.) That just isn't a reason, it seems to me. Nor is it a likely bit of dialogue. "Because it hurts" is both more likely to be said (if parents I know are typical) and more like a genuine reason not to pinch others. On p. 77 Joyce says that the theory of reasons he defends "understands reasons only as means to an agent's ends," but this seems misguided to me, given that the fact that an action's hurting others is a reason not to engage in that action. There is something wrong with you if you don't see this. (And if you don't see it then in a sense it is not part of your world, in which case your world is smaller than that of someone who does see it, which is an idea that has a Tractarian flavor to it.)
I think the following passage might be telling:
Of course, concerning Nazis we might not say "You ought not to have done that," for this sounds altogether too weak to capture the outrage -- rather, we appeal to the language of "evil" and "bestiality." But the "ought" statement is implied by the stronger language -- evil is, at the very least, something we ought not be.This suggests a scale or ladder (am I overdoing the Tractatus references yet?) with lesser evils at the bottom and greater ones near the top, or else a kind of onion of badness with mild evil in the center and greater evils on the outside, containing the lesser degrees within them. Language that does not really apply to great evils is then taken to be of the right kind but (merely) the wrong degree. I think this misses some of the richness, the diversity, of what can be recognized in our language. Joyce moves not only from milder language to stronger but also from the second person to the third. If you were talking to a Nazi would you tell them that what they had done was evil or bestial? You might, especially if you were struggling to find suitable words, but wouldn't these really be just as weak as saying "You ought not to have done that"? If you were to scold them I think you might have to resort to swearing or spitting, but scolding Nazis itself seems wrongheaded. I'm not sure there is anything to say. I think I would have nothing to say, anyway. The Nazis are such a paradigm of evil that attempting to express the evil of his ways to a Nazi would seem to be like attempting to express the redness of red or the obviousness of the most obvious truth one can think of.
This might not be true of an actual Nazi. Think of someone who joined the party but never took part in any of its crimes. He might feel that he really didn't do anything much wrong. There would be things you could say to someone like this, and things he might say in response. A real conversation could be had. Philosophers' Nazis are not like this. They are chosen because they are platonically evil. But then what does a sentence like "The Nazis were evil" mean? Not much, I think, unless it is being said to a child who is only just learning about the Nazis. The evil of the Nazis is like the immemorial age of the Earth, something that goes without saying. Except that it so goes without saying that it makes no sense to say it. By which I mean: it isn't clear what saying it would (could) be. What could be communicated by uttering the words "The Nazis were evil"? Well, anything you like. But what would be communicated thereby? It would, obviously, depend on the details of the situation, but it's hard to imagine circumstances in which these words told an adult something about the morality of the behavior we associate with the Nazis.
On p. 48 Joyce writes that "whatever kind of existence moral obligations have, it is a type which we are familiar with from other frameworks already accepted." So asking whether they exist is not like asking whether numbers exist, he thinks. And this is because moral obligations only exist if reasons for performing moral acts exist, and we know what reasons are. If x morally ought to phi, then x has a reason to phi, Joyce says. But this reason would typically be thought of as itself moral, wouldn't it? Hence not the same as regular reasons. So his attempt to avoid a possible Carnapian objection (roughly: within mathematics it is obvious that numbers exist; outside mathematics the question can only mean is mathematics worth bothering with; and the same goes for talk of moral obligations inside and outside morality) doesn't look as successful to me as it does to him.
I was going to talk about chapter 3 as well, but I think this is enough for today.