SQL Server merge and pk violation
I'm trying to understand why in a scenario like the following
DECLARE @source TABLE
(
orderId NVARCHAR(50),
customerId NVARCHAR(50)
)
DECLARE @target TABLE
(
orderId NVARCHAR(50) PRIMARY KEY,
customerId NVARCHAR(50) NOT NULL
)
INSERT INTO @source
VALUES ('test', '123'), ('test', '234')
MERGE @target AS TRG
USING (SELECT DISTINCT orderId, customerId
FROM @source) AS SRC ON SRC.orderId = TRG.orderId
WHEN MATCHED THEN
UPDATE SET TRG.customerId = SRC.customerId
WHEN NOT MATCHED BY TARGET THEN
INSERT (orderId, customerId)
VALUES (orderId, customerId);
I'm getting a duplicate key violation error:
Msg 2627, Level 14, State 1, Line 21
Violation of PRIMARY KEY constraint 'PK__#B3D7759__0809335D4BE1521F'. Cannot insert duplicate key in object 'dbo.@target'. The duplicate key value is (test).
What I expect is that the update statement finds the existing key and updates the customerId so that at the end I have in @target 1 row with orderId = 'test' and customerId = '234'.
For what I can assume, it instead tries to insert all records as it first doesn't find any key match at the beginning of the merge, causing the violation as the source contains the key multiple times.
Is this right? Is there any way to achieve what I am expecting using the merge function?
@user1443098
I've read your link, thanks. However I have a massive data insertion coming from a source table and going into 10 different tables; I tried to implement the procedure with a cursor and it took like 0.5s per record (with all the if exists statements). With merge statement, 300 rows have been inserted in the 10 different tables in less than one sec. So in my case it does a lot of difference in performance terms.
sql-server merge primary-key
|
show 6 more comments
I'm trying to understand why in a scenario like the following
DECLARE @source TABLE
(
orderId NVARCHAR(50),
customerId NVARCHAR(50)
)
DECLARE @target TABLE
(
orderId NVARCHAR(50) PRIMARY KEY,
customerId NVARCHAR(50) NOT NULL
)
INSERT INTO @source
VALUES ('test', '123'), ('test', '234')
MERGE @target AS TRG
USING (SELECT DISTINCT orderId, customerId
FROM @source) AS SRC ON SRC.orderId = TRG.orderId
WHEN MATCHED THEN
UPDATE SET TRG.customerId = SRC.customerId
WHEN NOT MATCHED BY TARGET THEN
INSERT (orderId, customerId)
VALUES (orderId, customerId);
I'm getting a duplicate key violation error:
Msg 2627, Level 14, State 1, Line 21
Violation of PRIMARY KEY constraint 'PK__#B3D7759__0809335D4BE1521F'. Cannot insert duplicate key in object 'dbo.@target'. The duplicate key value is (test).
What I expect is that the update statement finds the existing key and updates the customerId so that at the end I have in @target 1 row with orderId = 'test' and customerId = '234'.
For what I can assume, it instead tries to insert all records as it first doesn't find any key match at the beginning of the merge, causing the violation as the source contains the key multiple times.
Is this right? Is there any way to achieve what I am expecting using the merge function?
@user1443098
I've read your link, thanks. However I have a massive data insertion coming from a source table and going into 10 different tables; I tried to implement the procedure with a cursor and it took like 0.5s per record (with all the if exists statements). With merge statement, 300 rows have been inserted in the 10 different tables in less than one sec. So in my case it does a lot of difference in performance terms.
sql-server merge primary-key
2
There are many issues with MERGE. See mssqltips.com/sqlservertip/3074/… including key violations. I try not to use it
– user1443098
Nov 27 '18 at 21:47
1
@trg
starts out empty, so all of the rows in@src
will hit theNOT MATCHED
and you end up with two rows each withorderId
test
.MERGE
cannot touch rows twice -- or rather, it can, but whenever you've written aMERGE
that can potentially do that the results are nondeterministic and may error out. I agree with the right honorable 1443098 thatMERGE
should generally be avoided if the equivalent sequence ofINSERT
s andUPDATE
s is not too hard to write.
– Jeroen Mostert
Nov 27 '18 at 22:00
Considering thatOrderID
is the PK, and that's what you're matching on, what isUPDATE SET TRG.orderId = SRC.orderId
trying to achieve anyway?
– Larnu
Nov 27 '18 at 22:06
1
a really good question!
– tim
Nov 27 '18 at 22:13
Also, from the name@trg
, is it too presumptuous to assume this table will be filled by or otherwise involved in a trigger? Triggers are extra hard to get right, so avoid cleverness likeMERGE
in them like the plague.
– Jeroen Mostert
Nov 27 '18 at 22:17
|
show 6 more comments
I'm trying to understand why in a scenario like the following
DECLARE @source TABLE
(
orderId NVARCHAR(50),
customerId NVARCHAR(50)
)
DECLARE @target TABLE
(
orderId NVARCHAR(50) PRIMARY KEY,
customerId NVARCHAR(50) NOT NULL
)
INSERT INTO @source
VALUES ('test', '123'), ('test', '234')
MERGE @target AS TRG
USING (SELECT DISTINCT orderId, customerId
FROM @source) AS SRC ON SRC.orderId = TRG.orderId
WHEN MATCHED THEN
UPDATE SET TRG.customerId = SRC.customerId
WHEN NOT MATCHED BY TARGET THEN
INSERT (orderId, customerId)
VALUES (orderId, customerId);
I'm getting a duplicate key violation error:
Msg 2627, Level 14, State 1, Line 21
Violation of PRIMARY KEY constraint 'PK__#B3D7759__0809335D4BE1521F'. Cannot insert duplicate key in object 'dbo.@target'. The duplicate key value is (test).
What I expect is that the update statement finds the existing key and updates the customerId so that at the end I have in @target 1 row with orderId = 'test' and customerId = '234'.
For what I can assume, it instead tries to insert all records as it first doesn't find any key match at the beginning of the merge, causing the violation as the source contains the key multiple times.
Is this right? Is there any way to achieve what I am expecting using the merge function?
@user1443098
I've read your link, thanks. However I have a massive data insertion coming from a source table and going into 10 different tables; I tried to implement the procedure with a cursor and it took like 0.5s per record (with all the if exists statements). With merge statement, 300 rows have been inserted in the 10 different tables in less than one sec. So in my case it does a lot of difference in performance terms.
sql-server merge primary-key
I'm trying to understand why in a scenario like the following
DECLARE @source TABLE
(
orderId NVARCHAR(50),
customerId NVARCHAR(50)
)
DECLARE @target TABLE
(
orderId NVARCHAR(50) PRIMARY KEY,
customerId NVARCHAR(50) NOT NULL
)
INSERT INTO @source
VALUES ('test', '123'), ('test', '234')
MERGE @target AS TRG
USING (SELECT DISTINCT orderId, customerId
FROM @source) AS SRC ON SRC.orderId = TRG.orderId
WHEN MATCHED THEN
UPDATE SET TRG.customerId = SRC.customerId
WHEN NOT MATCHED BY TARGET THEN
INSERT (orderId, customerId)
VALUES (orderId, customerId);
I'm getting a duplicate key violation error:
Msg 2627, Level 14, State 1, Line 21
Violation of PRIMARY KEY constraint 'PK__#B3D7759__0809335D4BE1521F'. Cannot insert duplicate key in object 'dbo.@target'. The duplicate key value is (test).
What I expect is that the update statement finds the existing key and updates the customerId so that at the end I have in @target 1 row with orderId = 'test' and customerId = '234'.
For what I can assume, it instead tries to insert all records as it first doesn't find any key match at the beginning of the merge, causing the violation as the source contains the key multiple times.
Is this right? Is there any way to achieve what I am expecting using the merge function?
@user1443098
I've read your link, thanks. However I have a massive data insertion coming from a source table and going into 10 different tables; I tried to implement the procedure with a cursor and it took like 0.5s per record (with all the if exists statements). With merge statement, 300 rows have been inserted in the 10 different tables in less than one sec. So in my case it does a lot of difference in performance terms.
sql-server merge primary-key
sql-server merge primary-key
edited Nov 27 '18 at 22:35
Mark
asked Nov 27 '18 at 21:44
MarkMark
3291519
3291519
2
There are many issues with MERGE. See mssqltips.com/sqlservertip/3074/… including key violations. I try not to use it
– user1443098
Nov 27 '18 at 21:47
1
@trg
starts out empty, so all of the rows in@src
will hit theNOT MATCHED
and you end up with two rows each withorderId
test
.MERGE
cannot touch rows twice -- or rather, it can, but whenever you've written aMERGE
that can potentially do that the results are nondeterministic and may error out. I agree with the right honorable 1443098 thatMERGE
should generally be avoided if the equivalent sequence ofINSERT
s andUPDATE
s is not too hard to write.
– Jeroen Mostert
Nov 27 '18 at 22:00
Considering thatOrderID
is the PK, and that's what you're matching on, what isUPDATE SET TRG.orderId = SRC.orderId
trying to achieve anyway?
– Larnu
Nov 27 '18 at 22:06
1
a really good question!
– tim
Nov 27 '18 at 22:13
Also, from the name@trg
, is it too presumptuous to assume this table will be filled by or otherwise involved in a trigger? Triggers are extra hard to get right, so avoid cleverness likeMERGE
in them like the plague.
– Jeroen Mostert
Nov 27 '18 at 22:17
|
show 6 more comments
2
There are many issues with MERGE. See mssqltips.com/sqlservertip/3074/… including key violations. I try not to use it
– user1443098
Nov 27 '18 at 21:47
1
@trg
starts out empty, so all of the rows in@src
will hit theNOT MATCHED
and you end up with two rows each withorderId
test
.MERGE
cannot touch rows twice -- or rather, it can, but whenever you've written aMERGE
that can potentially do that the results are nondeterministic and may error out. I agree with the right honorable 1443098 thatMERGE
should generally be avoided if the equivalent sequence ofINSERT
s andUPDATE
s is not too hard to write.
– Jeroen Mostert
Nov 27 '18 at 22:00
Considering thatOrderID
is the PK, and that's what you're matching on, what isUPDATE SET TRG.orderId = SRC.orderId
trying to achieve anyway?
– Larnu
Nov 27 '18 at 22:06
1
a really good question!
– tim
Nov 27 '18 at 22:13
Also, from the name@trg
, is it too presumptuous to assume this table will be filled by or otherwise involved in a trigger? Triggers are extra hard to get right, so avoid cleverness likeMERGE
in them like the plague.
– Jeroen Mostert
Nov 27 '18 at 22:17
2
2
There are many issues with MERGE. See mssqltips.com/sqlservertip/3074/… including key violations. I try not to use it
– user1443098
Nov 27 '18 at 21:47
There are many issues with MERGE. See mssqltips.com/sqlservertip/3074/… including key violations. I try not to use it
– user1443098
Nov 27 '18 at 21:47
1
1
@trg
starts out empty, so all of the rows in @src
will hit the NOT MATCHED
and you end up with two rows each with orderId
test
. MERGE
cannot touch rows twice -- or rather, it can, but whenever you've written a MERGE
that can potentially do that the results are nondeterministic and may error out. I agree with the right honorable 1443098 that MERGE
should generally be avoided if the equivalent sequence of INSERT
s and UPDATE
s is not too hard to write.– Jeroen Mostert
Nov 27 '18 at 22:00
@trg
starts out empty, so all of the rows in @src
will hit the NOT MATCHED
and you end up with two rows each with orderId
test
. MERGE
cannot touch rows twice -- or rather, it can, but whenever you've written a MERGE
that can potentially do that the results are nondeterministic and may error out. I agree with the right honorable 1443098 that MERGE
should generally be avoided if the equivalent sequence of INSERT
s and UPDATE
s is not too hard to write.– Jeroen Mostert
Nov 27 '18 at 22:00
Considering that
OrderID
is the PK, and that's what you're matching on, what is UPDATE SET TRG.orderId = SRC.orderId
trying to achieve anyway?– Larnu
Nov 27 '18 at 22:06
Considering that
OrderID
is the PK, and that's what you're matching on, what is UPDATE SET TRG.orderId = SRC.orderId
trying to achieve anyway?– Larnu
Nov 27 '18 at 22:06
1
1
a really good question!
– tim
Nov 27 '18 at 22:13
a really good question!
– tim
Nov 27 '18 at 22:13
Also, from the name
@trg
, is it too presumptuous to assume this table will be filled by or otherwise involved in a trigger? Triggers are extra hard to get right, so avoid cleverness like MERGE
in them like the plague.– Jeroen Mostert
Nov 27 '18 at 22:17
Also, from the name
@trg
, is it too presumptuous to assume this table will be filled by or otherwise involved in a trigger? Triggers are extra hard to get right, so avoid cleverness like MERGE
in them like the plague.– Jeroen Mostert
Nov 27 '18 at 22:17
|
show 6 more comments
1 Answer
1
active
oldest
votes
There are two records in @source
with the same OrderID
. There is not a match for either record in @target
so the NOT MATCHED
clause is trying to insert both of these records. But it can not do this because the primary key on OrderID
in the @target
table requires that all inserted records have unique values for OrderID
. The duplication of values into the primary key causes the primary key violation.
If you are expecting duplicates are possible in the source... you should eliminate them in your USING
sub-query. Something like this:
(SELECT orderId, max(customerId) customerId
FROM @source
group by orderId)
Ok thanks, so that's how I've assumed. I was hoping the statement was designed to handle this cases. That's really a shame because I've found it very good under the performance aspect.
– Mark
Nov 27 '18 at 22:38
You can eliminate the possibility of duplicates in your source... I added an example to the answer.
– Brian Pressler
Nov 27 '18 at 22:43
Yes thanks, I was thinking to do something on the source. The downside is that I have to forecast all passible key violations in advance and treat them.
– Mark
Nov 27 '18 at 22:48
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53508654%2fsql-server-merge-and-pk-violation%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
There are two records in @source
with the same OrderID
. There is not a match for either record in @target
so the NOT MATCHED
clause is trying to insert both of these records. But it can not do this because the primary key on OrderID
in the @target
table requires that all inserted records have unique values for OrderID
. The duplication of values into the primary key causes the primary key violation.
If you are expecting duplicates are possible in the source... you should eliminate them in your USING
sub-query. Something like this:
(SELECT orderId, max(customerId) customerId
FROM @source
group by orderId)
Ok thanks, so that's how I've assumed. I was hoping the statement was designed to handle this cases. That's really a shame because I've found it very good under the performance aspect.
– Mark
Nov 27 '18 at 22:38
You can eliminate the possibility of duplicates in your source... I added an example to the answer.
– Brian Pressler
Nov 27 '18 at 22:43
Yes thanks, I was thinking to do something on the source. The downside is that I have to forecast all passible key violations in advance and treat them.
– Mark
Nov 27 '18 at 22:48
add a comment |
There are two records in @source
with the same OrderID
. There is not a match for either record in @target
so the NOT MATCHED
clause is trying to insert both of these records. But it can not do this because the primary key on OrderID
in the @target
table requires that all inserted records have unique values for OrderID
. The duplication of values into the primary key causes the primary key violation.
If you are expecting duplicates are possible in the source... you should eliminate them in your USING
sub-query. Something like this:
(SELECT orderId, max(customerId) customerId
FROM @source
group by orderId)
Ok thanks, so that's how I've assumed. I was hoping the statement was designed to handle this cases. That's really a shame because I've found it very good under the performance aspect.
– Mark
Nov 27 '18 at 22:38
You can eliminate the possibility of duplicates in your source... I added an example to the answer.
– Brian Pressler
Nov 27 '18 at 22:43
Yes thanks, I was thinking to do something on the source. The downside is that I have to forecast all passible key violations in advance and treat them.
– Mark
Nov 27 '18 at 22:48
add a comment |
There are two records in @source
with the same OrderID
. There is not a match for either record in @target
so the NOT MATCHED
clause is trying to insert both of these records. But it can not do this because the primary key on OrderID
in the @target
table requires that all inserted records have unique values for OrderID
. The duplication of values into the primary key causes the primary key violation.
If you are expecting duplicates are possible in the source... you should eliminate them in your USING
sub-query. Something like this:
(SELECT orderId, max(customerId) customerId
FROM @source
group by orderId)
There are two records in @source
with the same OrderID
. There is not a match for either record in @target
so the NOT MATCHED
clause is trying to insert both of these records. But it can not do this because the primary key on OrderID
in the @target
table requires that all inserted records have unique values for OrderID
. The duplication of values into the primary key causes the primary key violation.
If you are expecting duplicates are possible in the source... you should eliminate them in your USING
sub-query. Something like this:
(SELECT orderId, max(customerId) customerId
FROM @source
group by orderId)
edited Nov 27 '18 at 23:02
answered Nov 27 '18 at 22:02
Brian PresslerBrian Pressler
5,82111038
5,82111038
Ok thanks, so that's how I've assumed. I was hoping the statement was designed to handle this cases. That's really a shame because I've found it very good under the performance aspect.
– Mark
Nov 27 '18 at 22:38
You can eliminate the possibility of duplicates in your source... I added an example to the answer.
– Brian Pressler
Nov 27 '18 at 22:43
Yes thanks, I was thinking to do something on the source. The downside is that I have to forecast all passible key violations in advance and treat them.
– Mark
Nov 27 '18 at 22:48
add a comment |
Ok thanks, so that's how I've assumed. I was hoping the statement was designed to handle this cases. That's really a shame because I've found it very good under the performance aspect.
– Mark
Nov 27 '18 at 22:38
You can eliminate the possibility of duplicates in your source... I added an example to the answer.
– Brian Pressler
Nov 27 '18 at 22:43
Yes thanks, I was thinking to do something on the source. The downside is that I have to forecast all passible key violations in advance and treat them.
– Mark
Nov 27 '18 at 22:48
Ok thanks, so that's how I've assumed. I was hoping the statement was designed to handle this cases. That's really a shame because I've found it very good under the performance aspect.
– Mark
Nov 27 '18 at 22:38
Ok thanks, so that's how I've assumed. I was hoping the statement was designed to handle this cases. That's really a shame because I've found it very good under the performance aspect.
– Mark
Nov 27 '18 at 22:38
You can eliminate the possibility of duplicates in your source... I added an example to the answer.
– Brian Pressler
Nov 27 '18 at 22:43
You can eliminate the possibility of duplicates in your source... I added an example to the answer.
– Brian Pressler
Nov 27 '18 at 22:43
Yes thanks, I was thinking to do something on the source. The downside is that I have to forecast all passible key violations in advance and treat them.
– Mark
Nov 27 '18 at 22:48
Yes thanks, I was thinking to do something on the source. The downside is that I have to forecast all passible key violations in advance and treat them.
– Mark
Nov 27 '18 at 22:48
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53508654%2fsql-server-merge-and-pk-violation%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
There are many issues with MERGE. See mssqltips.com/sqlservertip/3074/… including key violations. I try not to use it
– user1443098
Nov 27 '18 at 21:47
1
@trg
starts out empty, so all of the rows in@src
will hit theNOT MATCHED
and you end up with two rows each withorderId
test
.MERGE
cannot touch rows twice -- or rather, it can, but whenever you've written aMERGE
that can potentially do that the results are nondeterministic and may error out. I agree with the right honorable 1443098 thatMERGE
should generally be avoided if the equivalent sequence ofINSERT
s andUPDATE
s is not too hard to write.– Jeroen Mostert
Nov 27 '18 at 22:00
Considering that
OrderID
is the PK, and that's what you're matching on, what isUPDATE SET TRG.orderId = SRC.orderId
trying to achieve anyway?– Larnu
Nov 27 '18 at 22:06
1
a really good question!
– tim
Nov 27 '18 at 22:13
Also, from the name
@trg
, is it too presumptuous to assume this table will be filled by or otherwise involved in a trigger? Triggers are extra hard to get right, so avoid cleverness likeMERGE
in them like the plague.– Jeroen Mostert
Nov 27 '18 at 22:17