Delete duplicate records from a SQL table without a primary key
I have the below table with the below records in it
create table employee
(
EmpId number,
EmpName varchar2(10),
EmpSSN varchar2(11)
);
insert into employee values(1, 'Jack', '555-55-5555');
insert into employee values (2, 'Joe', '555-56-5555');
insert into employee values (3, 'Fred', '555-57-5555');
insert into employee values (4, 'Mike', '555-58-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6, 'Lisa', '555-70-5555');
insert into employee values (1, 'Jack', '555-55-5555');
insert into employee values (4, 'Mike', '555-58-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6 ,'Lisa', '555-70-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6, 'Lisa', '555-70-5555');
I dont have any primary key in this table .But i have the above records in my table already.
I want to remove the duplicate records which has the same value in EmpId and EmpSSN fields.
Ex : Emp id 5
Can any one help me to frame a query to delete those duplicate records
Thanks in advance
sql sql-server-2005 tsql duplicate-removal
|
show 1 more comment
I have the below table with the below records in it
create table employee
(
EmpId number,
EmpName varchar2(10),
EmpSSN varchar2(11)
);
insert into employee values(1, 'Jack', '555-55-5555');
insert into employee values (2, 'Joe', '555-56-5555');
insert into employee values (3, 'Fred', '555-57-5555');
insert into employee values (4, 'Mike', '555-58-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6, 'Lisa', '555-70-5555');
insert into employee values (1, 'Jack', '555-55-5555');
insert into employee values (4, 'Mike', '555-58-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6 ,'Lisa', '555-70-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6, 'Lisa', '555-70-5555');
I dont have any primary key in this table .But i have the above records in my table already.
I want to remove the duplicate records which has the same value in EmpId and EmpSSN fields.
Ex : Emp id 5
Can any one help me to frame a query to delete those duplicate records
Thanks in advance
sql sql-server-2005 tsql duplicate-removal
Can you ADD a primary key?? What database system or you using? Oracle? Please specify so in your question!
– marc_s
Jun 12 '09 at 7:19
2
What if it has the same EmpID and EmpSSn, but different names?
– cjk
Jun 12 '09 at 7:24
its SQL server 2005
– Shyju
Jun 12 '09 at 8:02
We don't have varchar2 in SQL Server, any version
– gbn
Jun 12 '09 at 10:14
Hmmm... neither "number" nor "varchar2" are valid SQL Server 2005 data types.... smells like Oracle to me.
– marc_s
Jun 12 '09 at 10:52
|
show 1 more comment
I have the below table with the below records in it
create table employee
(
EmpId number,
EmpName varchar2(10),
EmpSSN varchar2(11)
);
insert into employee values(1, 'Jack', '555-55-5555');
insert into employee values (2, 'Joe', '555-56-5555');
insert into employee values (3, 'Fred', '555-57-5555');
insert into employee values (4, 'Mike', '555-58-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6, 'Lisa', '555-70-5555');
insert into employee values (1, 'Jack', '555-55-5555');
insert into employee values (4, 'Mike', '555-58-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6 ,'Lisa', '555-70-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6, 'Lisa', '555-70-5555');
I dont have any primary key in this table .But i have the above records in my table already.
I want to remove the duplicate records which has the same value in EmpId and EmpSSN fields.
Ex : Emp id 5
Can any one help me to frame a query to delete those duplicate records
Thanks in advance
sql sql-server-2005 tsql duplicate-removal
I have the below table with the below records in it
create table employee
(
EmpId number,
EmpName varchar2(10),
EmpSSN varchar2(11)
);
insert into employee values(1, 'Jack', '555-55-5555');
insert into employee values (2, 'Joe', '555-56-5555');
insert into employee values (3, 'Fred', '555-57-5555');
insert into employee values (4, 'Mike', '555-58-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6, 'Lisa', '555-70-5555');
insert into employee values (1, 'Jack', '555-55-5555');
insert into employee values (4, 'Mike', '555-58-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6 ,'Lisa', '555-70-5555');
insert into employee values (5, 'Cathy', '555-59-5555');
insert into employee values (6, 'Lisa', '555-70-5555');
I dont have any primary key in this table .But i have the above records in my table already.
I want to remove the duplicate records which has the same value in EmpId and EmpSSN fields.
Ex : Emp id 5
Can any one help me to frame a query to delete those duplicate records
Thanks in advance
sql sql-server-2005 tsql duplicate-removal
sql sql-server-2005 tsql duplicate-removal
edited Jul 16 '10 at 7:47
abatishchev
70.2k70265397
70.2k70265397
asked Jun 12 '09 at 7:12
ShyjuShyju
147k87333443
147k87333443
Can you ADD a primary key?? What database system or you using? Oracle? Please specify so in your question!
– marc_s
Jun 12 '09 at 7:19
2
What if it has the same EmpID and EmpSSn, but different names?
– cjk
Jun 12 '09 at 7:24
its SQL server 2005
– Shyju
Jun 12 '09 at 8:02
We don't have varchar2 in SQL Server, any version
– gbn
Jun 12 '09 at 10:14
Hmmm... neither "number" nor "varchar2" are valid SQL Server 2005 data types.... smells like Oracle to me.
– marc_s
Jun 12 '09 at 10:52
|
show 1 more comment
Can you ADD a primary key?? What database system or you using? Oracle? Please specify so in your question!
– marc_s
Jun 12 '09 at 7:19
2
What if it has the same EmpID and EmpSSn, but different names?
– cjk
Jun 12 '09 at 7:24
its SQL server 2005
– Shyju
Jun 12 '09 at 8:02
We don't have varchar2 in SQL Server, any version
– gbn
Jun 12 '09 at 10:14
Hmmm... neither "number" nor "varchar2" are valid SQL Server 2005 data types.... smells like Oracle to me.
– marc_s
Jun 12 '09 at 10:52
Can you ADD a primary key?? What database system or you using? Oracle? Please specify so in your question!
– marc_s
Jun 12 '09 at 7:19
Can you ADD a primary key?? What database system or you using? Oracle? Please specify so in your question!
– marc_s
Jun 12 '09 at 7:19
2
2
What if it has the same EmpID and EmpSSn, but different names?
– cjk
Jun 12 '09 at 7:24
What if it has the same EmpID and EmpSSn, but different names?
– cjk
Jun 12 '09 at 7:24
its SQL server 2005
– Shyju
Jun 12 '09 at 8:02
its SQL server 2005
– Shyju
Jun 12 '09 at 8:02
We don't have varchar2 in SQL Server, any version
– gbn
Jun 12 '09 at 10:14
We don't have varchar2 in SQL Server, any version
– gbn
Jun 12 '09 at 10:14
Hmmm... neither "number" nor "varchar2" are valid SQL Server 2005 data types.... smells like Oracle to me.
– marc_s
Jun 12 '09 at 10:52
Hmmm... neither "number" nor "varchar2" are valid SQL Server 2005 data types.... smells like Oracle to me.
– marc_s
Jun 12 '09 at 10:52
|
show 1 more comment
18 Answers
18
active
oldest
votes
Add a Primary Key (code below)
Run the correct delete (code below)
Consider WHY you woudln't want to keep that primary key.
Assuming MSSQL or compatible:
ALTER TABLE Employee ADD EmployeeID int identity(1,1) PRIMARY KEY;
WHILE EXISTS (SELECT COUNT(*) FROM Employee GROUP BY EmpID, EmpSSN HAVING COUNT(*) > 1)
BEGIN
DELETE FROM Employee WHERE EmployeeID IN
(
SELECT MIN(EmployeeID) as [DeleteID]
FROM Employee
GROUP BY EmpID, EmpSSN
HAVING COUNT(*) > 1
)
END
7
+1: to quote some SQL god: "if it doesn't have a primary key, it's not a table"
– marc_s
Jun 12 '09 at 7:26
2
+1 A primary key identifies a row. No PK = no sense. @marc_s: a clustered index differentiates a table from a heap. No PK simply means no data integrity
– gbn
Jun 12 '09 at 7:28
@gbn: even a heap is considered a table :-) This quote was more along the lines: unless you specify a primary key, a table really doesn't have much usefulness (except in edge cases like bulk import / temporary tables etc.)
– marc_s
Jun 12 '09 at 7:57
even in those edge cases I almost always add a primary key, just so I can delete dupped recrds if need be.
– HLGEM
Jun 12 '09 at 17:05
1
Looks like the duplicate removal is being done so the EmpID can be the primary key. The other data seems dependant on it.
– Stu Pegg
Oct 6 '09 at 11:59
|
show 2 more comments
It is very simple. I tried in SQL Server 2008
DELETE SUB FROM
(SELECT ROW_NUMBER() OVER (PARTITION BY EmpId, EmpName, EmpSSN ORDER BY EmpId) cnt
FROM Employee) SUB
WHERE SUB.cnt > 1
2
+1 Also works fine in 2005
– Martin Smith
Nov 7 '11 at 12:58
2
This works well when you have a lot of columns to group by, and it neatly deals with the NULL != NULL when comparing two columns. You don't have to list each column twice like some of the other answers ("a.col = b.col" type thing), and even more importantly, you don't have to check "((a.col = b.col) OR (a.col IS NULL AND b.col IS NULL))" on NULL columns.
– Bryce Wagner
Apr 3 '13 at 19:58
4
This answer actually resolves the problem, without structural changes. Works perfectly.
– WhizBang
Jan 28 '14 at 17:50
add a comment |
Use the row number to differentiate between duplicate records. Keep the first row number for an EmpID/EmpSSN and delete the rest:
DELETE FROM Employee a
WHERE ROW_NUMBER() <> ( SELECT MIN( ROW_NUMBER() )
FROM Employee b
WHERE a.EmpID = b.EmpID
AND a.EmpSSN = b.EmpSSN )
3
+1 A good solution to avoid having to make structural changes
– Stu Pegg
Oct 6 '09 at 11:56
Will it work for Oracle? I had this issue stackoverflow.com/questions/34948301/…
– Rudziankoŭ
Jan 27 '16 at 12:50
add a comment |
With duplicates
As
(Select *, ROW_NUMBER() Over (PARTITION by EmpID,EmpSSN Order by EmpID,EmpSSN) as Duplicate From Employee)
delete From duplicates
Where Duplicate > 1 ;
This will update Table and remove all duplicates from the Table!
add a comment |
select distinct * into newtablename from oldtablename
Now, the newtablename
will have no duplicate records.
Simply change the table name(newtablename
) by pressing F2 in object explorer in sql server.
add a comment |
You could create a temporary table #tempemployee
containing a select distinct
of your employee
table.
Then delete from employee
.
Then insert into employee select from #tempemployee
.
Like Josh said - even if you know the duplicates, deleting them will be impossile since you cannot actually refer to a specific record if it is an exact duplicate of another record.
2
Only trick there is if the names are different but the ID/SSN match. You'd have to somehow pick one because distinct wouldn't help there.
– Josh
Jun 12 '09 at 7:19
1
+1 this is the most straightforward and portable solution. OP does not state what brand of database he uses.
– Bill Karwin
Jun 12 '09 at 7:22
@Josh: from the OP's sample, it looks like that's not an issue. The duplicate rows are identical in all columns.
– Bill Karwin
Jun 12 '09 at 7:23
add a comment |
Code
DELETE DUP
FROM
(
SELECT ROW_NUMBER() OVER (PARTITION BY Clientid ORDER BY Clientid ) AS Val
FROM ClientMaster
) DUP
WHERE DUP.Val > 1
Explanation
Use an inner query to construct a view over the table which includes a field based on Row_Number()
, partitioned by those columns you wish to be unique.
Delete from the results of this inner query, selecting anything which does not have a row number of 1; i.e. the duplicates; not the original.
The order by
clause of the row_number window function is needed for a valid syntax; you can put any column name here. If you wish to change which of the results is treated as a duplicate (e.g. keep the earliest or most recent, etc), then the column(s) used here do matter; i.e. you want to specify the order such that the record you wish to keep will come first in the result.
Welcome to Stack Overflow! Code only answers are not very useful on their own. It would help if you could add some detail explaining how/why it answers the question.
– SiHa
Sep 27 '16 at 15:24
add a comment |
If you don't want to create a new primary key you can use the TOP command in SQL Server:
declare @ID int
while EXISTS(select count(*) from Employee group by EmpId having count(*)> 1)
begin
select top 1 @ID = EmpId
from Employee
group by EmpId
having count(*) > 1
DELETE TOP(1) FROM Employee WHERE EmpId = @ID
end
add a comment |
ITS easy use below query
WITH Dups AS
(
SELECT col1,col2,col3,
ROW_NUMBER() OVER(PARTITION BY col1,col2,col3 ORDER BY (SELECT 0)) AS rn
FROM mytable
)
DELETE FROM Dups WHERE rn > 1
add a comment |
I'm not an SQL expert so bear with me. I'm sure you'll get a better answer soon enough. Here's how you can find the duplicate records.
select t1.empid, t1.empssn, count(*)
from employee as t1
inner join employee as t2 on (t1.empid=t2.empid and t1.empssn = t2.empssn)
group by t1.empid, t1.empssn
having count(*) > 1
Deleting them will be more tricky because there is nothing in the data that you could use in a delete statement to differentiate the duplicates. I suspect the answer will involve row_number() or adding an identity column.
add a comment |
create unique clustered index Employee_idx
on Employee ( EmpId,EmpSSN )
with ignore_dup_key
You can drop the index if you don't need it.
add a comment |
no ID
, no rowcount()
or no temp table
needed....
WHILE
(
SELECT COUNT(*)
FROM TBLEMP
WHERE EMPNO
IN (SELECT empno from tblemp group by empno having count(empno)>1)) > 1
DELETE top(1)
FROM TBLEMP
WHERE EMPNO IN (SELECT empno from tblemp group by empno having count(empno)>1)
add a comment |
there are two columns in the a table ID and name where names are repeating with different IDs so for that you may use this query:
.
.
DELETE FROM dbo.tbl1
WHERE id NOT IN (
Select MIN(Id) AS namecount FROM tbl1
GROUP BY Name
)
add a comment |
Having a database table without Primary Key is really and will say extremely BAD PRACTICE...so after you add one (ALTER TABLE)
Run this until you don't see any more duplicated records (that is the purpose of HAVING COUNT)
DELETE FROM [TABLE_NAME] WHERE [Id] IN
(
SELECT MAX([Id])
FROM [TABLE_NAME]
GROUP BY [TARGET_COLUMN]
HAVING COUNT(*) > 1
)
SELECT MAX([Id]),[TABLE_NAME], COUNT(*) AS dupeCount
FROM [TABLE_NAME]
GROUP BY [TABLE_NAME]
HAVING COUNT(*) > 1
MAX([Id]) will cause to delete latest records (ones added after first created) in case you want the opposite meaning that in case of requiring deleting first records and leave the last record inserted please use MIN([Id])
add a comment |
delete sub from (select ROW_NUMBER() OVer(Partition by empid order by empid)cnt from employee)sub
where sub.cnt>1
Welcome to stackoverflow. This is an old question with a well established answer. IF you believe your answer adds something significant and new, please expand it with more explanation.
– Simon.S.A.
Nov 28 '18 at 3:45
add a comment |
select t1.* from employee t1, employee t2 where t1.empid=t2.empid and t1.empname = t2.empname and t1.salary = t2.salary
group by t1.empid, t1.empname,t1.salary having count(*) > 1
Which question does it answer?
– Rudziankoŭ
Jan 27 '16 at 12:58
add a comment |
DELETE FROM 'test'
USING 'test' , 'test' as vtable
WHERE test.id>vtable.id and test.common_column=vtable.common_column
Using this we can remove duplicate records
add a comment |
ALTER IGNORE TABLE test
ADD UNIQUE INDEX 'test' ('b');
@ here 'b' is column name to uniqueness,
@ here 'test' is index name.
1
Not remotely valid SQL Server syntax.
– Martin Smith
Nov 7 '11 at 14:39
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f985384%2fdelete-duplicate-records-from-a-sql-table-without-a-primary-key%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
18 Answers
18
active
oldest
votes
18 Answers
18
active
oldest
votes
active
oldest
votes
active
oldest
votes
Add a Primary Key (code below)
Run the correct delete (code below)
Consider WHY you woudln't want to keep that primary key.
Assuming MSSQL or compatible:
ALTER TABLE Employee ADD EmployeeID int identity(1,1) PRIMARY KEY;
WHILE EXISTS (SELECT COUNT(*) FROM Employee GROUP BY EmpID, EmpSSN HAVING COUNT(*) > 1)
BEGIN
DELETE FROM Employee WHERE EmployeeID IN
(
SELECT MIN(EmployeeID) as [DeleteID]
FROM Employee
GROUP BY EmpID, EmpSSN
HAVING COUNT(*) > 1
)
END
7
+1: to quote some SQL god: "if it doesn't have a primary key, it's not a table"
– marc_s
Jun 12 '09 at 7:26
2
+1 A primary key identifies a row. No PK = no sense. @marc_s: a clustered index differentiates a table from a heap. No PK simply means no data integrity
– gbn
Jun 12 '09 at 7:28
@gbn: even a heap is considered a table :-) This quote was more along the lines: unless you specify a primary key, a table really doesn't have much usefulness (except in edge cases like bulk import / temporary tables etc.)
– marc_s
Jun 12 '09 at 7:57
even in those edge cases I almost always add a primary key, just so I can delete dupped recrds if need be.
– HLGEM
Jun 12 '09 at 17:05
1
Looks like the duplicate removal is being done so the EmpID can be the primary key. The other data seems dependant on it.
– Stu Pegg
Oct 6 '09 at 11:59
|
show 2 more comments
Add a Primary Key (code below)
Run the correct delete (code below)
Consider WHY you woudln't want to keep that primary key.
Assuming MSSQL or compatible:
ALTER TABLE Employee ADD EmployeeID int identity(1,1) PRIMARY KEY;
WHILE EXISTS (SELECT COUNT(*) FROM Employee GROUP BY EmpID, EmpSSN HAVING COUNT(*) > 1)
BEGIN
DELETE FROM Employee WHERE EmployeeID IN
(
SELECT MIN(EmployeeID) as [DeleteID]
FROM Employee
GROUP BY EmpID, EmpSSN
HAVING COUNT(*) > 1
)
END
7
+1: to quote some SQL god: "if it doesn't have a primary key, it's not a table"
– marc_s
Jun 12 '09 at 7:26
2
+1 A primary key identifies a row. No PK = no sense. @marc_s: a clustered index differentiates a table from a heap. No PK simply means no data integrity
– gbn
Jun 12 '09 at 7:28
@gbn: even a heap is considered a table :-) This quote was more along the lines: unless you specify a primary key, a table really doesn't have much usefulness (except in edge cases like bulk import / temporary tables etc.)
– marc_s
Jun 12 '09 at 7:57
even in those edge cases I almost always add a primary key, just so I can delete dupped recrds if need be.
– HLGEM
Jun 12 '09 at 17:05
1
Looks like the duplicate removal is being done so the EmpID can be the primary key. The other data seems dependant on it.
– Stu Pegg
Oct 6 '09 at 11:59
|
show 2 more comments
Add a Primary Key (code below)
Run the correct delete (code below)
Consider WHY you woudln't want to keep that primary key.
Assuming MSSQL or compatible:
ALTER TABLE Employee ADD EmployeeID int identity(1,1) PRIMARY KEY;
WHILE EXISTS (SELECT COUNT(*) FROM Employee GROUP BY EmpID, EmpSSN HAVING COUNT(*) > 1)
BEGIN
DELETE FROM Employee WHERE EmployeeID IN
(
SELECT MIN(EmployeeID) as [DeleteID]
FROM Employee
GROUP BY EmpID, EmpSSN
HAVING COUNT(*) > 1
)
END
Add a Primary Key (code below)
Run the correct delete (code below)
Consider WHY you woudln't want to keep that primary key.
Assuming MSSQL or compatible:
ALTER TABLE Employee ADD EmployeeID int identity(1,1) PRIMARY KEY;
WHILE EXISTS (SELECT COUNT(*) FROM Employee GROUP BY EmpID, EmpSSN HAVING COUNT(*) > 1)
BEGIN
DELETE FROM Employee WHERE EmployeeID IN
(
SELECT MIN(EmployeeID) as [DeleteID]
FROM Employee
GROUP BY EmpID, EmpSSN
HAVING COUNT(*) > 1
)
END
edited Jul 16 '10 at 7:47
abatishchev
70.2k70265397
70.2k70265397
answered Jun 12 '09 at 7:23
cjkcjk
39.4k67099
39.4k67099
7
+1: to quote some SQL god: "if it doesn't have a primary key, it's not a table"
– marc_s
Jun 12 '09 at 7:26
2
+1 A primary key identifies a row. No PK = no sense. @marc_s: a clustered index differentiates a table from a heap. No PK simply means no data integrity
– gbn
Jun 12 '09 at 7:28
@gbn: even a heap is considered a table :-) This quote was more along the lines: unless you specify a primary key, a table really doesn't have much usefulness (except in edge cases like bulk import / temporary tables etc.)
– marc_s
Jun 12 '09 at 7:57
even in those edge cases I almost always add a primary key, just so I can delete dupped recrds if need be.
– HLGEM
Jun 12 '09 at 17:05
1
Looks like the duplicate removal is being done so the EmpID can be the primary key. The other data seems dependant on it.
– Stu Pegg
Oct 6 '09 at 11:59
|
show 2 more comments
7
+1: to quote some SQL god: "if it doesn't have a primary key, it's not a table"
– marc_s
Jun 12 '09 at 7:26
2
+1 A primary key identifies a row. No PK = no sense. @marc_s: a clustered index differentiates a table from a heap. No PK simply means no data integrity
– gbn
Jun 12 '09 at 7:28
@gbn: even a heap is considered a table :-) This quote was more along the lines: unless you specify a primary key, a table really doesn't have much usefulness (except in edge cases like bulk import / temporary tables etc.)
– marc_s
Jun 12 '09 at 7:57
even in those edge cases I almost always add a primary key, just so I can delete dupped recrds if need be.
– HLGEM
Jun 12 '09 at 17:05
1
Looks like the duplicate removal is being done so the EmpID can be the primary key. The other data seems dependant on it.
– Stu Pegg
Oct 6 '09 at 11:59
7
7
+1: to quote some SQL god: "if it doesn't have a primary key, it's not a table"
– marc_s
Jun 12 '09 at 7:26
+1: to quote some SQL god: "if it doesn't have a primary key, it's not a table"
– marc_s
Jun 12 '09 at 7:26
2
2
+1 A primary key identifies a row. No PK = no sense. @marc_s: a clustered index differentiates a table from a heap. No PK simply means no data integrity
– gbn
Jun 12 '09 at 7:28
+1 A primary key identifies a row. No PK = no sense. @marc_s: a clustered index differentiates a table from a heap. No PK simply means no data integrity
– gbn
Jun 12 '09 at 7:28
@gbn: even a heap is considered a table :-) This quote was more along the lines: unless you specify a primary key, a table really doesn't have much usefulness (except in edge cases like bulk import / temporary tables etc.)
– marc_s
Jun 12 '09 at 7:57
@gbn: even a heap is considered a table :-) This quote was more along the lines: unless you specify a primary key, a table really doesn't have much usefulness (except in edge cases like bulk import / temporary tables etc.)
– marc_s
Jun 12 '09 at 7:57
even in those edge cases I almost always add a primary key, just so I can delete dupped recrds if need be.
– HLGEM
Jun 12 '09 at 17:05
even in those edge cases I almost always add a primary key, just so I can delete dupped recrds if need be.
– HLGEM
Jun 12 '09 at 17:05
1
1
Looks like the duplicate removal is being done so the EmpID can be the primary key. The other data seems dependant on it.
– Stu Pegg
Oct 6 '09 at 11:59
Looks like the duplicate removal is being done so the EmpID can be the primary key. The other data seems dependant on it.
– Stu Pegg
Oct 6 '09 at 11:59
|
show 2 more comments
It is very simple. I tried in SQL Server 2008
DELETE SUB FROM
(SELECT ROW_NUMBER() OVER (PARTITION BY EmpId, EmpName, EmpSSN ORDER BY EmpId) cnt
FROM Employee) SUB
WHERE SUB.cnt > 1
2
+1 Also works fine in 2005
– Martin Smith
Nov 7 '11 at 12:58
2
This works well when you have a lot of columns to group by, and it neatly deals with the NULL != NULL when comparing two columns. You don't have to list each column twice like some of the other answers ("a.col = b.col" type thing), and even more importantly, you don't have to check "((a.col = b.col) OR (a.col IS NULL AND b.col IS NULL))" on NULL columns.
– Bryce Wagner
Apr 3 '13 at 19:58
4
This answer actually resolves the problem, without structural changes. Works perfectly.
– WhizBang
Jan 28 '14 at 17:50
add a comment |
It is very simple. I tried in SQL Server 2008
DELETE SUB FROM
(SELECT ROW_NUMBER() OVER (PARTITION BY EmpId, EmpName, EmpSSN ORDER BY EmpId) cnt
FROM Employee) SUB
WHERE SUB.cnt > 1
2
+1 Also works fine in 2005
– Martin Smith
Nov 7 '11 at 12:58
2
This works well when you have a lot of columns to group by, and it neatly deals with the NULL != NULL when comparing two columns. You don't have to list each column twice like some of the other answers ("a.col = b.col" type thing), and even more importantly, you don't have to check "((a.col = b.col) OR (a.col IS NULL AND b.col IS NULL))" on NULL columns.
– Bryce Wagner
Apr 3 '13 at 19:58
4
This answer actually resolves the problem, without structural changes. Works perfectly.
– WhizBang
Jan 28 '14 at 17:50
add a comment |
It is very simple. I tried in SQL Server 2008
DELETE SUB FROM
(SELECT ROW_NUMBER() OVER (PARTITION BY EmpId, EmpName, EmpSSN ORDER BY EmpId) cnt
FROM Employee) SUB
WHERE SUB.cnt > 1
It is very simple. I tried in SQL Server 2008
DELETE SUB FROM
(SELECT ROW_NUMBER() OVER (PARTITION BY EmpId, EmpName, EmpSSN ORDER BY EmpId) cnt
FROM Employee) SUB
WHERE SUB.cnt > 1
edited Jun 16 '14 at 18:03
Flea
8,89656068
8,89656068
answered Sep 12 '11 at 12:22
Anjib RajkhowaAnjib Rajkhowa
71152
71152
2
+1 Also works fine in 2005
– Martin Smith
Nov 7 '11 at 12:58
2
This works well when you have a lot of columns to group by, and it neatly deals with the NULL != NULL when comparing two columns. You don't have to list each column twice like some of the other answers ("a.col = b.col" type thing), and even more importantly, you don't have to check "((a.col = b.col) OR (a.col IS NULL AND b.col IS NULL))" on NULL columns.
– Bryce Wagner
Apr 3 '13 at 19:58
4
This answer actually resolves the problem, without structural changes. Works perfectly.
– WhizBang
Jan 28 '14 at 17:50
add a comment |
2
+1 Also works fine in 2005
– Martin Smith
Nov 7 '11 at 12:58
2
This works well when you have a lot of columns to group by, and it neatly deals with the NULL != NULL when comparing two columns. You don't have to list each column twice like some of the other answers ("a.col = b.col" type thing), and even more importantly, you don't have to check "((a.col = b.col) OR (a.col IS NULL AND b.col IS NULL))" on NULL columns.
– Bryce Wagner
Apr 3 '13 at 19:58
4
This answer actually resolves the problem, without structural changes. Works perfectly.
– WhizBang
Jan 28 '14 at 17:50
2
2
+1 Also works fine in 2005
– Martin Smith
Nov 7 '11 at 12:58
+1 Also works fine in 2005
– Martin Smith
Nov 7 '11 at 12:58
2
2
This works well when you have a lot of columns to group by, and it neatly deals with the NULL != NULL when comparing two columns. You don't have to list each column twice like some of the other answers ("a.col = b.col" type thing), and even more importantly, you don't have to check "((a.col = b.col) OR (a.col IS NULL AND b.col IS NULL))" on NULL columns.
– Bryce Wagner
Apr 3 '13 at 19:58
This works well when you have a lot of columns to group by, and it neatly deals with the NULL != NULL when comparing two columns. You don't have to list each column twice like some of the other answers ("a.col = b.col" type thing), and even more importantly, you don't have to check "((a.col = b.col) OR (a.col IS NULL AND b.col IS NULL))" on NULL columns.
– Bryce Wagner
Apr 3 '13 at 19:58
4
4
This answer actually resolves the problem, without structural changes. Works perfectly.
– WhizBang
Jan 28 '14 at 17:50
This answer actually resolves the problem, without structural changes. Works perfectly.
– WhizBang
Jan 28 '14 at 17:50
add a comment |
Use the row number to differentiate between duplicate records. Keep the first row number for an EmpID/EmpSSN and delete the rest:
DELETE FROM Employee a
WHERE ROW_NUMBER() <> ( SELECT MIN( ROW_NUMBER() )
FROM Employee b
WHERE a.EmpID = b.EmpID
AND a.EmpSSN = b.EmpSSN )
3
+1 A good solution to avoid having to make structural changes
– Stu Pegg
Oct 6 '09 at 11:56
Will it work for Oracle? I had this issue stackoverflow.com/questions/34948301/…
– Rudziankoŭ
Jan 27 '16 at 12:50
add a comment |
Use the row number to differentiate between duplicate records. Keep the first row number for an EmpID/EmpSSN and delete the rest:
DELETE FROM Employee a
WHERE ROW_NUMBER() <> ( SELECT MIN( ROW_NUMBER() )
FROM Employee b
WHERE a.EmpID = b.EmpID
AND a.EmpSSN = b.EmpSSN )
3
+1 A good solution to avoid having to make structural changes
– Stu Pegg
Oct 6 '09 at 11:56
Will it work for Oracle? I had this issue stackoverflow.com/questions/34948301/…
– Rudziankoŭ
Jan 27 '16 at 12:50
add a comment |
Use the row number to differentiate between duplicate records. Keep the first row number for an EmpID/EmpSSN and delete the rest:
DELETE FROM Employee a
WHERE ROW_NUMBER() <> ( SELECT MIN( ROW_NUMBER() )
FROM Employee b
WHERE a.EmpID = b.EmpID
AND a.EmpSSN = b.EmpSSN )
Use the row number to differentiate between duplicate records. Keep the first row number for an EmpID/EmpSSN and delete the rest:
DELETE FROM Employee a
WHERE ROW_NUMBER() <> ( SELECT MIN( ROW_NUMBER() )
FROM Employee b
WHERE a.EmpID = b.EmpID
AND a.EmpSSN = b.EmpSSN )
edited Jul 20 '10 at 1:33
answered Jun 12 '09 at 17:01
Paul MorganPaul Morgan
19.7k31927
19.7k31927
3
+1 A good solution to avoid having to make structural changes
– Stu Pegg
Oct 6 '09 at 11:56
Will it work for Oracle? I had this issue stackoverflow.com/questions/34948301/…
– Rudziankoŭ
Jan 27 '16 at 12:50
add a comment |
3
+1 A good solution to avoid having to make structural changes
– Stu Pegg
Oct 6 '09 at 11:56
Will it work for Oracle? I had this issue stackoverflow.com/questions/34948301/…
– Rudziankoŭ
Jan 27 '16 at 12:50
3
3
+1 A good solution to avoid having to make structural changes
– Stu Pegg
Oct 6 '09 at 11:56
+1 A good solution to avoid having to make structural changes
– Stu Pegg
Oct 6 '09 at 11:56
Will it work for Oracle? I had this issue stackoverflow.com/questions/34948301/…
– Rudziankoŭ
Jan 27 '16 at 12:50
Will it work for Oracle? I had this issue stackoverflow.com/questions/34948301/…
– Rudziankoŭ
Jan 27 '16 at 12:50
add a comment |
With duplicates
As
(Select *, ROW_NUMBER() Over (PARTITION by EmpID,EmpSSN Order by EmpID,EmpSSN) as Duplicate From Employee)
delete From duplicates
Where Duplicate > 1 ;
This will update Table and remove all duplicates from the Table!
add a comment |
With duplicates
As
(Select *, ROW_NUMBER() Over (PARTITION by EmpID,EmpSSN Order by EmpID,EmpSSN) as Duplicate From Employee)
delete From duplicates
Where Duplicate > 1 ;
This will update Table and remove all duplicates from the Table!
add a comment |
With duplicates
As
(Select *, ROW_NUMBER() Over (PARTITION by EmpID,EmpSSN Order by EmpID,EmpSSN) as Duplicate From Employee)
delete From duplicates
Where Duplicate > 1 ;
This will update Table and remove all duplicates from the Table!
With duplicates
As
(Select *, ROW_NUMBER() Over (PARTITION by EmpID,EmpSSN Order by EmpID,EmpSSN) as Duplicate From Employee)
delete From duplicates
Where Duplicate > 1 ;
This will update Table and remove all duplicates from the Table!
edited Dec 6 '11 at 16:44
John Conde
186k80373427
186k80373427
answered Dec 6 '11 at 16:38
Nirav ParikhNirav Parikh
9111
9111
add a comment |
add a comment |
select distinct * into newtablename from oldtablename
Now, the newtablename
will have no duplicate records.
Simply change the table name(newtablename
) by pressing F2 in object explorer in sql server.
add a comment |
select distinct * into newtablename from oldtablename
Now, the newtablename
will have no duplicate records.
Simply change the table name(newtablename
) by pressing F2 in object explorer in sql server.
add a comment |
select distinct * into newtablename from oldtablename
Now, the newtablename
will have no duplicate records.
Simply change the table name(newtablename
) by pressing F2 in object explorer in sql server.
select distinct * into newtablename from oldtablename
Now, the newtablename
will have no duplicate records.
Simply change the table name(newtablename
) by pressing F2 in object explorer in sql server.
edited Oct 20 '12 at 1:38
askmish
5,5091638
5,5091638
answered Jun 20 '12 at 11:57
naga vara prasadnaga vara prasad
7111
7111
add a comment |
add a comment |
You could create a temporary table #tempemployee
containing a select distinct
of your employee
table.
Then delete from employee
.
Then insert into employee select from #tempemployee
.
Like Josh said - even if you know the duplicates, deleting them will be impossile since you cannot actually refer to a specific record if it is an exact duplicate of another record.
2
Only trick there is if the names are different but the ID/SSN match. You'd have to somehow pick one because distinct wouldn't help there.
– Josh
Jun 12 '09 at 7:19
1
+1 this is the most straightforward and portable solution. OP does not state what brand of database he uses.
– Bill Karwin
Jun 12 '09 at 7:22
@Josh: from the OP's sample, it looks like that's not an issue. The duplicate rows are identical in all columns.
– Bill Karwin
Jun 12 '09 at 7:23
add a comment |
You could create a temporary table #tempemployee
containing a select distinct
of your employee
table.
Then delete from employee
.
Then insert into employee select from #tempemployee
.
Like Josh said - even if you know the duplicates, deleting them will be impossile since you cannot actually refer to a specific record if it is an exact duplicate of another record.
2
Only trick there is if the names are different but the ID/SSN match. You'd have to somehow pick one because distinct wouldn't help there.
– Josh
Jun 12 '09 at 7:19
1
+1 this is the most straightforward and portable solution. OP does not state what brand of database he uses.
– Bill Karwin
Jun 12 '09 at 7:22
@Josh: from the OP's sample, it looks like that's not an issue. The duplicate rows are identical in all columns.
– Bill Karwin
Jun 12 '09 at 7:23
add a comment |
You could create a temporary table #tempemployee
containing a select distinct
of your employee
table.
Then delete from employee
.
Then insert into employee select from #tempemployee
.
Like Josh said - even if you know the duplicates, deleting them will be impossile since you cannot actually refer to a specific record if it is an exact duplicate of another record.
You could create a temporary table #tempemployee
containing a select distinct
of your employee
table.
Then delete from employee
.
Then insert into employee select from #tempemployee
.
Like Josh said - even if you know the duplicates, deleting them will be impossile since you cannot actually refer to a specific record if it is an exact duplicate of another record.
answered Jun 12 '09 at 7:16
Daren ThomasDaren Thomas
43.1k37131184
43.1k37131184
2
Only trick there is if the names are different but the ID/SSN match. You'd have to somehow pick one because distinct wouldn't help there.
– Josh
Jun 12 '09 at 7:19
1
+1 this is the most straightforward and portable solution. OP does not state what brand of database he uses.
– Bill Karwin
Jun 12 '09 at 7:22
@Josh: from the OP's sample, it looks like that's not an issue. The duplicate rows are identical in all columns.
– Bill Karwin
Jun 12 '09 at 7:23
add a comment |
2
Only trick there is if the names are different but the ID/SSN match. You'd have to somehow pick one because distinct wouldn't help there.
– Josh
Jun 12 '09 at 7:19
1
+1 this is the most straightforward and portable solution. OP does not state what brand of database he uses.
– Bill Karwin
Jun 12 '09 at 7:22
@Josh: from the OP's sample, it looks like that's not an issue. The duplicate rows are identical in all columns.
– Bill Karwin
Jun 12 '09 at 7:23
2
2
Only trick there is if the names are different but the ID/SSN match. You'd have to somehow pick one because distinct wouldn't help there.
– Josh
Jun 12 '09 at 7:19
Only trick there is if the names are different but the ID/SSN match. You'd have to somehow pick one because distinct wouldn't help there.
– Josh
Jun 12 '09 at 7:19
1
1
+1 this is the most straightforward and portable solution. OP does not state what brand of database he uses.
– Bill Karwin
Jun 12 '09 at 7:22
+1 this is the most straightforward and portable solution. OP does not state what brand of database he uses.
– Bill Karwin
Jun 12 '09 at 7:22
@Josh: from the OP's sample, it looks like that's not an issue. The duplicate rows are identical in all columns.
– Bill Karwin
Jun 12 '09 at 7:23
@Josh: from the OP's sample, it looks like that's not an issue. The duplicate rows are identical in all columns.
– Bill Karwin
Jun 12 '09 at 7:23
add a comment |
Code
DELETE DUP
FROM
(
SELECT ROW_NUMBER() OVER (PARTITION BY Clientid ORDER BY Clientid ) AS Val
FROM ClientMaster
) DUP
WHERE DUP.Val > 1
Explanation
Use an inner query to construct a view over the table which includes a field based on Row_Number()
, partitioned by those columns you wish to be unique.
Delete from the results of this inner query, selecting anything which does not have a row number of 1; i.e. the duplicates; not the original.
The order by
clause of the row_number window function is needed for a valid syntax; you can put any column name here. If you wish to change which of the results is treated as a duplicate (e.g. keep the earliest or most recent, etc), then the column(s) used here do matter; i.e. you want to specify the order such that the record you wish to keep will come first in the result.
Welcome to Stack Overflow! Code only answers are not very useful on their own. It would help if you could add some detail explaining how/why it answers the question.
– SiHa
Sep 27 '16 at 15:24
add a comment |
Code
DELETE DUP
FROM
(
SELECT ROW_NUMBER() OVER (PARTITION BY Clientid ORDER BY Clientid ) AS Val
FROM ClientMaster
) DUP
WHERE DUP.Val > 1
Explanation
Use an inner query to construct a view over the table which includes a field based on Row_Number()
, partitioned by those columns you wish to be unique.
Delete from the results of this inner query, selecting anything which does not have a row number of 1; i.e. the duplicates; not the original.
The order by
clause of the row_number window function is needed for a valid syntax; you can put any column name here. If you wish to change which of the results is treated as a duplicate (e.g. keep the earliest or most recent, etc), then the column(s) used here do matter; i.e. you want to specify the order such that the record you wish to keep will come first in the result.
Welcome to Stack Overflow! Code only answers are not very useful on their own. It would help if you could add some detail explaining how/why it answers the question.
– SiHa
Sep 27 '16 at 15:24
add a comment |
Code
DELETE DUP
FROM
(
SELECT ROW_NUMBER() OVER (PARTITION BY Clientid ORDER BY Clientid ) AS Val
FROM ClientMaster
) DUP
WHERE DUP.Val > 1
Explanation
Use an inner query to construct a view over the table which includes a field based on Row_Number()
, partitioned by those columns you wish to be unique.
Delete from the results of this inner query, selecting anything which does not have a row number of 1; i.e. the duplicates; not the original.
The order by
clause of the row_number window function is needed for a valid syntax; you can put any column name here. If you wish to change which of the results is treated as a duplicate (e.g. keep the earliest or most recent, etc), then the column(s) used here do matter; i.e. you want to specify the order such that the record you wish to keep will come first in the result.
Code
DELETE DUP
FROM
(
SELECT ROW_NUMBER() OVER (PARTITION BY Clientid ORDER BY Clientid ) AS Val
FROM ClientMaster
) DUP
WHERE DUP.Val > 1
Explanation
Use an inner query to construct a view over the table which includes a field based on Row_Number()
, partitioned by those columns you wish to be unique.
Delete from the results of this inner query, selecting anything which does not have a row number of 1; i.e. the duplicates; not the original.
The order by
clause of the row_number window function is needed for a valid syntax; you can put any column name here. If you wish to change which of the results is treated as a duplicate (e.g. keep the earliest or most recent, etc), then the column(s) used here do matter; i.e. you want to specify the order such that the record you wish to keep will come first in the result.
edited Apr 7 '17 at 11:39
JohnLBevan
14.6k146111
14.6k146111
answered Sep 27 '16 at 6:40
kamz kamarajankamz kamarajan
413
413
Welcome to Stack Overflow! Code only answers are not very useful on their own. It would help if you could add some detail explaining how/why it answers the question.
– SiHa
Sep 27 '16 at 15:24
add a comment |
Welcome to Stack Overflow! Code only answers are not very useful on their own. It would help if you could add some detail explaining how/why it answers the question.
– SiHa
Sep 27 '16 at 15:24
Welcome to Stack Overflow! Code only answers are not very useful on their own. It would help if you could add some detail explaining how/why it answers the question.
– SiHa
Sep 27 '16 at 15:24
Welcome to Stack Overflow! Code only answers are not very useful on their own. It would help if you could add some detail explaining how/why it answers the question.
– SiHa
Sep 27 '16 at 15:24
add a comment |
If you don't want to create a new primary key you can use the TOP command in SQL Server:
declare @ID int
while EXISTS(select count(*) from Employee group by EmpId having count(*)> 1)
begin
select top 1 @ID = EmpId
from Employee
group by EmpId
having count(*) > 1
DELETE TOP(1) FROM Employee WHERE EmpId = @ID
end
add a comment |
If you don't want to create a new primary key you can use the TOP command in SQL Server:
declare @ID int
while EXISTS(select count(*) from Employee group by EmpId having count(*)> 1)
begin
select top 1 @ID = EmpId
from Employee
group by EmpId
having count(*) > 1
DELETE TOP(1) FROM Employee WHERE EmpId = @ID
end
add a comment |
If you don't want to create a new primary key you can use the TOP command in SQL Server:
declare @ID int
while EXISTS(select count(*) from Employee group by EmpId having count(*)> 1)
begin
select top 1 @ID = EmpId
from Employee
group by EmpId
having count(*) > 1
DELETE TOP(1) FROM Employee WHERE EmpId = @ID
end
If you don't want to create a new primary key you can use the TOP command in SQL Server:
declare @ID int
while EXISTS(select count(*) from Employee group by EmpId having count(*)> 1)
begin
select top 1 @ID = EmpId
from Employee
group by EmpId
having count(*) > 1
DELETE TOP(1) FROM Employee WHERE EmpId = @ID
end
answered Jun 2 '10 at 21:30
JoeJoe
32633
32633
add a comment |
add a comment |
ITS easy use below query
WITH Dups AS
(
SELECT col1,col2,col3,
ROW_NUMBER() OVER(PARTITION BY col1,col2,col3 ORDER BY (SELECT 0)) AS rn
FROM mytable
)
DELETE FROM Dups WHERE rn > 1
add a comment |
ITS easy use below query
WITH Dups AS
(
SELECT col1,col2,col3,
ROW_NUMBER() OVER(PARTITION BY col1,col2,col3 ORDER BY (SELECT 0)) AS rn
FROM mytable
)
DELETE FROM Dups WHERE rn > 1
add a comment |
ITS easy use below query
WITH Dups AS
(
SELECT col1,col2,col3,
ROW_NUMBER() OVER(PARTITION BY col1,col2,col3 ORDER BY (SELECT 0)) AS rn
FROM mytable
)
DELETE FROM Dups WHERE rn > 1
ITS easy use below query
WITH Dups AS
(
SELECT col1,col2,col3,
ROW_NUMBER() OVER(PARTITION BY col1,col2,col3 ORDER BY (SELECT 0)) AS rn
FROM mytable
)
DELETE FROM Dups WHERE rn > 1
answered Sep 19 '16 at 10:20
Abhishek JaiswalAbhishek Jaiswal
71184
71184
add a comment |
add a comment |
I'm not an SQL expert so bear with me. I'm sure you'll get a better answer soon enough. Here's how you can find the duplicate records.
select t1.empid, t1.empssn, count(*)
from employee as t1
inner join employee as t2 on (t1.empid=t2.empid and t1.empssn = t2.empssn)
group by t1.empid, t1.empssn
having count(*) > 1
Deleting them will be more tricky because there is nothing in the data that you could use in a delete statement to differentiate the duplicates. I suspect the answer will involve row_number() or adding an identity column.
add a comment |
I'm not an SQL expert so bear with me. I'm sure you'll get a better answer soon enough. Here's how you can find the duplicate records.
select t1.empid, t1.empssn, count(*)
from employee as t1
inner join employee as t2 on (t1.empid=t2.empid and t1.empssn = t2.empssn)
group by t1.empid, t1.empssn
having count(*) > 1
Deleting them will be more tricky because there is nothing in the data that you could use in a delete statement to differentiate the duplicates. I suspect the answer will involve row_number() or adding an identity column.
add a comment |
I'm not an SQL expert so bear with me. I'm sure you'll get a better answer soon enough. Here's how you can find the duplicate records.
select t1.empid, t1.empssn, count(*)
from employee as t1
inner join employee as t2 on (t1.empid=t2.empid and t1.empssn = t2.empssn)
group by t1.empid, t1.empssn
having count(*) > 1
Deleting them will be more tricky because there is nothing in the data that you could use in a delete statement to differentiate the duplicates. I suspect the answer will involve row_number() or adding an identity column.
I'm not an SQL expert so bear with me. I'm sure you'll get a better answer soon enough. Here's how you can find the duplicate records.
select t1.empid, t1.empssn, count(*)
from employee as t1
inner join employee as t2 on (t1.empid=t2.empid and t1.empssn = t2.empssn)
group by t1.empid, t1.empssn
having count(*) > 1
Deleting them will be more tricky because there is nothing in the data that you could use in a delete statement to differentiate the duplicates. I suspect the answer will involve row_number() or adding an identity column.
answered Jun 12 '09 at 7:18
JoshJosh
58.1k14120143
58.1k14120143
add a comment |
add a comment |
create unique clustered index Employee_idx
on Employee ( EmpId,EmpSSN )
with ignore_dup_key
You can drop the index if you don't need it.
add a comment |
create unique clustered index Employee_idx
on Employee ( EmpId,EmpSSN )
with ignore_dup_key
You can drop the index if you don't need it.
add a comment |
create unique clustered index Employee_idx
on Employee ( EmpId,EmpSSN )
with ignore_dup_key
You can drop the index if you don't need it.
create unique clustered index Employee_idx
on Employee ( EmpId,EmpSSN )
with ignore_dup_key
You can drop the index if you don't need it.
edited Jul 6 '12 at 11:35
Anil
343222
343222
answered Jul 16 '10 at 7:49
SudhirSudhir
1
1
add a comment |
add a comment |
no ID
, no rowcount()
or no temp table
needed....
WHILE
(
SELECT COUNT(*)
FROM TBLEMP
WHERE EMPNO
IN (SELECT empno from tblemp group by empno having count(empno)>1)) > 1
DELETE top(1)
FROM TBLEMP
WHERE EMPNO IN (SELECT empno from tblemp group by empno having count(empno)>1)
add a comment |
no ID
, no rowcount()
or no temp table
needed....
WHILE
(
SELECT COUNT(*)
FROM TBLEMP
WHERE EMPNO
IN (SELECT empno from tblemp group by empno having count(empno)>1)) > 1
DELETE top(1)
FROM TBLEMP
WHERE EMPNO IN (SELECT empno from tblemp group by empno having count(empno)>1)
add a comment |
no ID
, no rowcount()
or no temp table
needed....
WHILE
(
SELECT COUNT(*)
FROM TBLEMP
WHERE EMPNO
IN (SELECT empno from tblemp group by empno having count(empno)>1)) > 1
DELETE top(1)
FROM TBLEMP
WHERE EMPNO IN (SELECT empno from tblemp group by empno having count(empno)>1)
no ID
, no rowcount()
or no temp table
needed....
WHILE
(
SELECT COUNT(*)
FROM TBLEMP
WHERE EMPNO
IN (SELECT empno from tblemp group by empno having count(empno)>1)) > 1
DELETE top(1)
FROM TBLEMP
WHERE EMPNO IN (SELECT empno from tblemp group by empno having count(empno)>1)
edited Apr 14 '13 at 6:25
Praveen Nambiar
3,90311526
3,90311526
answered Apr 14 '13 at 5:56
Ashish SahuAshish Sahu
511
511
add a comment |
add a comment |
there are two columns in the a table ID and name where names are repeating with different IDs so for that you may use this query:
.
.
DELETE FROM dbo.tbl1
WHERE id NOT IN (
Select MIN(Id) AS namecount FROM tbl1
GROUP BY Name
)
add a comment |
there are two columns in the a table ID and name where names are repeating with different IDs so for that you may use this query:
.
.
DELETE FROM dbo.tbl1
WHERE id NOT IN (
Select MIN(Id) AS namecount FROM tbl1
GROUP BY Name
)
add a comment |
there are two columns in the a table ID and name where names are repeating with different IDs so for that you may use this query:
.
.
DELETE FROM dbo.tbl1
WHERE id NOT IN (
Select MIN(Id) AS namecount FROM tbl1
GROUP BY Name
)
there are two columns in the a table ID and name where names are repeating with different IDs so for that you may use this query:
.
.
DELETE FROM dbo.tbl1
WHERE id NOT IN (
Select MIN(Id) AS namecount FROM tbl1
GROUP BY Name
)
edited Jun 18 '13 at 13:55
Jens Kloster
8,93243250
8,93243250
answered Jun 18 '13 at 13:35
user2497372user2497372
1
1
add a comment |
add a comment |
Having a database table without Primary Key is really and will say extremely BAD PRACTICE...so after you add one (ALTER TABLE)
Run this until you don't see any more duplicated records (that is the purpose of HAVING COUNT)
DELETE FROM [TABLE_NAME] WHERE [Id] IN
(
SELECT MAX([Id])
FROM [TABLE_NAME]
GROUP BY [TARGET_COLUMN]
HAVING COUNT(*) > 1
)
SELECT MAX([Id]),[TABLE_NAME], COUNT(*) AS dupeCount
FROM [TABLE_NAME]
GROUP BY [TABLE_NAME]
HAVING COUNT(*) > 1
MAX([Id]) will cause to delete latest records (ones added after first created) in case you want the opposite meaning that in case of requiring deleting first records and leave the last record inserted please use MIN([Id])
add a comment |
Having a database table without Primary Key is really and will say extremely BAD PRACTICE...so after you add one (ALTER TABLE)
Run this until you don't see any more duplicated records (that is the purpose of HAVING COUNT)
DELETE FROM [TABLE_NAME] WHERE [Id] IN
(
SELECT MAX([Id])
FROM [TABLE_NAME]
GROUP BY [TARGET_COLUMN]
HAVING COUNT(*) > 1
)
SELECT MAX([Id]),[TABLE_NAME], COUNT(*) AS dupeCount
FROM [TABLE_NAME]
GROUP BY [TABLE_NAME]
HAVING COUNT(*) > 1
MAX([Id]) will cause to delete latest records (ones added after first created) in case you want the opposite meaning that in case of requiring deleting first records and leave the last record inserted please use MIN([Id])
add a comment |
Having a database table without Primary Key is really and will say extremely BAD PRACTICE...so after you add one (ALTER TABLE)
Run this until you don't see any more duplicated records (that is the purpose of HAVING COUNT)
DELETE FROM [TABLE_NAME] WHERE [Id] IN
(
SELECT MAX([Id])
FROM [TABLE_NAME]
GROUP BY [TARGET_COLUMN]
HAVING COUNT(*) > 1
)
SELECT MAX([Id]),[TABLE_NAME], COUNT(*) AS dupeCount
FROM [TABLE_NAME]
GROUP BY [TABLE_NAME]
HAVING COUNT(*) > 1
MAX([Id]) will cause to delete latest records (ones added after first created) in case you want the opposite meaning that in case of requiring deleting first records and leave the last record inserted please use MIN([Id])
Having a database table without Primary Key is really and will say extremely BAD PRACTICE...so after you add one (ALTER TABLE)
Run this until you don't see any more duplicated records (that is the purpose of HAVING COUNT)
DELETE FROM [TABLE_NAME] WHERE [Id] IN
(
SELECT MAX([Id])
FROM [TABLE_NAME]
GROUP BY [TARGET_COLUMN]
HAVING COUNT(*) > 1
)
SELECT MAX([Id]),[TABLE_NAME], COUNT(*) AS dupeCount
FROM [TABLE_NAME]
GROUP BY [TABLE_NAME]
HAVING COUNT(*) > 1
MAX([Id]) will cause to delete latest records (ones added after first created) in case you want the opposite meaning that in case of requiring deleting first records and leave the last record inserted please use MIN([Id])
answered Jul 19 '14 at 4:08
d1jhoni1bd1jhoni1b
4,3242727
4,3242727
add a comment |
add a comment |
delete sub from (select ROW_NUMBER() OVer(Partition by empid order by empid)cnt from employee)sub
where sub.cnt>1
Welcome to stackoverflow. This is an old question with a well established answer. IF you believe your answer adds something significant and new, please expand it with more explanation.
– Simon.S.A.
Nov 28 '18 at 3:45
add a comment |
delete sub from (select ROW_NUMBER() OVer(Partition by empid order by empid)cnt from employee)sub
where sub.cnt>1
Welcome to stackoverflow. This is an old question with a well established answer. IF you believe your answer adds something significant and new, please expand it with more explanation.
– Simon.S.A.
Nov 28 '18 at 3:45
add a comment |
delete sub from (select ROW_NUMBER() OVer(Partition by empid order by empid)cnt from employee)sub
where sub.cnt>1
delete sub from (select ROW_NUMBER() OVer(Partition by empid order by empid)cnt from employee)sub
where sub.cnt>1
answered Nov 28 '18 at 3:24
Sudhar PSudhar P
1
1
Welcome to stackoverflow. This is an old question with a well established answer. IF you believe your answer adds something significant and new, please expand it with more explanation.
– Simon.S.A.
Nov 28 '18 at 3:45
add a comment |
Welcome to stackoverflow. This is an old question with a well established answer. IF you believe your answer adds something significant and new, please expand it with more explanation.
– Simon.S.A.
Nov 28 '18 at 3:45
Welcome to stackoverflow. This is an old question with a well established answer. IF you believe your answer adds something significant and new, please expand it with more explanation.
– Simon.S.A.
Nov 28 '18 at 3:45
Welcome to stackoverflow. This is an old question with a well established answer. IF you believe your answer adds something significant and new, please expand it with more explanation.
– Simon.S.A.
Nov 28 '18 at 3:45
add a comment |
select t1.* from employee t1, employee t2 where t1.empid=t2.empid and t1.empname = t2.empname and t1.salary = t2.salary
group by t1.empid, t1.empname,t1.salary having count(*) > 1
Which question does it answer?
– Rudziankoŭ
Jan 27 '16 at 12:58
add a comment |
select t1.* from employee t1, employee t2 where t1.empid=t2.empid and t1.empname = t2.empname and t1.salary = t2.salary
group by t1.empid, t1.empname,t1.salary having count(*) > 1
Which question does it answer?
– Rudziankoŭ
Jan 27 '16 at 12:58
add a comment |
select t1.* from employee t1, employee t2 where t1.empid=t2.empid and t1.empname = t2.empname and t1.salary = t2.salary
group by t1.empid, t1.empname,t1.salary having count(*) > 1
select t1.* from employee t1, employee t2 where t1.empid=t2.empid and t1.empname = t2.empname and t1.salary = t2.salary
group by t1.empid, t1.empname,t1.salary having count(*) > 1
edited Jun 3 '13 at 9:38
Freelancer
7,70873273
7,70873273
answered Oct 6 '09 at 11:52
tejas rupani
Which question does it answer?
– Rudziankoŭ
Jan 27 '16 at 12:58
add a comment |
Which question does it answer?
– Rudziankoŭ
Jan 27 '16 at 12:58
Which question does it answer?
– Rudziankoŭ
Jan 27 '16 at 12:58
Which question does it answer?
– Rudziankoŭ
Jan 27 '16 at 12:58
add a comment |
DELETE FROM 'test'
USING 'test' , 'test' as vtable
WHERE test.id>vtable.id and test.common_column=vtable.common_column
Using this we can remove duplicate records
add a comment |
DELETE FROM 'test'
USING 'test' , 'test' as vtable
WHERE test.id>vtable.id and test.common_column=vtable.common_column
Using this we can remove duplicate records
add a comment |
DELETE FROM 'test'
USING 'test' , 'test' as vtable
WHERE test.id>vtable.id and test.common_column=vtable.common_column
Using this we can remove duplicate records
DELETE FROM 'test'
USING 'test' , 'test' as vtable
WHERE test.id>vtable.id and test.common_column=vtable.common_column
Using this we can remove duplicate records
edited Nov 7 '11 at 12:53
The_Fox
6,41123263
6,41123263
answered Nov 9 '10 at 11:04
jayaram.pagotijayaram.pagoti
1
1
add a comment |
add a comment |
ALTER IGNORE TABLE test
ADD UNIQUE INDEX 'test' ('b');
@ here 'b' is column name to uniqueness,
@ here 'test' is index name.
1
Not remotely valid SQL Server syntax.
– Martin Smith
Nov 7 '11 at 14:39
add a comment |
ALTER IGNORE TABLE test
ADD UNIQUE INDEX 'test' ('b');
@ here 'b' is column name to uniqueness,
@ here 'test' is index name.
1
Not remotely valid SQL Server syntax.
– Martin Smith
Nov 7 '11 at 14:39
add a comment |
ALTER IGNORE TABLE test
ADD UNIQUE INDEX 'test' ('b');
@ here 'b' is column name to uniqueness,
@ here 'test' is index name.
ALTER IGNORE TABLE test
ADD UNIQUE INDEX 'test' ('b');
@ here 'b' is column name to uniqueness,
@ here 'test' is index name.
edited Aug 1 '12 at 9:27
Anil
343222
343222
answered Nov 9 '10 at 10:18
jayaram.pagotijayaram.pagoti
1
1
1
Not remotely valid SQL Server syntax.
– Martin Smith
Nov 7 '11 at 14:39
add a comment |
1
Not remotely valid SQL Server syntax.
– Martin Smith
Nov 7 '11 at 14:39
1
1
Not remotely valid SQL Server syntax.
– Martin Smith
Nov 7 '11 at 14:39
Not remotely valid SQL Server syntax.
– Martin Smith
Nov 7 '11 at 14:39
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f985384%2fdelete-duplicate-records-from-a-sql-table-without-a-primary-key%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Can you ADD a primary key?? What database system or you using? Oracle? Please specify so in your question!
– marc_s
Jun 12 '09 at 7:19
2
What if it has the same EmpID and EmpSSn, but different names?
– cjk
Jun 12 '09 at 7:24
its SQL server 2005
– Shyju
Jun 12 '09 at 8:02
We don't have varchar2 in SQL Server, any version
– gbn
Jun 12 '09 at 10:14
Hmmm... neither "number" nor "varchar2" are valid SQL Server 2005 data types.... smells like Oracle to me.
– marc_s
Jun 12 '09 at 10:52